[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:47:48 dist-test-slave-pww0 krb5kdc[10088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:47:48 dist-test-slave-pww0 krb5kdc[10088](info): set up 2 sockets
Aug 11 18:47:48 dist-test-slave-pww0 krb5kdc[10088](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 18:47:49.513021 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:47:49 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:47:49.650377 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43413
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:43413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:49.793372 10117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:49.793623 10117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:49.793659 10117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:49.795002 10117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:47:49.795051 10117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:49.795069 10117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:49.795087 10117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:47:49.795102 10117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:47:49.797189 10117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:43413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:49.797595 10117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:49.797854 10117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:49.804239 10125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:49.804301 10117 server_base.cc:1034] running on GCE node
W20260811 18:47:49.804339 10128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:49.804481 10124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:49.804765 10117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:49.805176 10117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:49.806351 10117 hybrid_clock.cc:648] HybridClock initialized: now 1786474069806330 us; error 49 us; skew 500 ppm
Aug 11 18:47:49 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474069, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:49.809265 10117 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:47:49.809881 10117 webserver.cc:460] Webserver started at http://127.9.215.62:44229/ using document root <none> and password file <none>
I20260811 18:47:49.810141 10117 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:49.810195 10117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:49.810348 10117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:49.811739 10117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "86340b2de01848d79219a9305b651a9e"
format_stamp: "Formatted at 2026-08-11 18:47:49 on dist-test-slave-pww0"
server_key: "00c1052c82e292ca721472c64ea78ccc"
server_key_iv: "f89d59fb33e5c1379cf3bcd395c9b404"
server_key_version: "encryptionkey@0"
I20260811 18:47:49.812170 10117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "86340b2de01848d79219a9305b651a9e"
format_stamp: "Formatted at 2026-08-11 18:47:49 on dist-test-slave-pww0"
server_key: "00c1052c82e292ca721472c64ea78ccc"
server_key_iv: "f89d59fb33e5c1379cf3bcd395c9b404"
server_key_version: "encryptionkey@0"
I20260811 18:47:49.814404 10117 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:47:49.815491 10140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:49.815776 10117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:49.815898 10117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "86340b2de01848d79219a9305b651a9e"
format_stamp: "Formatted at 2026-08-11 18:47:49 on dist-test-slave-pww0"
server_key: "00c1052c82e292ca721472c64ea78ccc"
server_key_iv: "f89d59fb33e5c1379cf3bcd395c9b404"
server_key_version: "encryptionkey@0"
I20260811 18:47:49.815992 10117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:49.839928 10117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:49.843000 10117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:49.843201 10117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:49.857401 10117 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:43413
I20260811 18:47:49.858053 10117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:47:49.858003 10247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:43413 every 8 connection(s)
I20260811 18:47:49.859473 10250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:49.859540 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10117
I20260811 18:47:49.859651 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:47:49.860684 10076 external_mini_cluster.cc:1428] Setting key 2aeb2f06a8c8b8e0583e58ec648da6e6
I20260811 18:47:49.865264 10250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e: Bootstrap starting.
I20260811 18:47:49.867192 10250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:49.868232 10250 log.cc:824] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:49.869365 10250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e: No bootstrap required, opened a new log
I20260811 18:47:49.874966 10250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86340b2de01848d79219a9305b651a9e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43413 } }
I20260811 18:47:49.875200 10250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:49.875298 10250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86340b2de01848d79219a9305b651a9e, State: Initialized, Role: FOLLOWER
I20260811 18:47:49.875504 10250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86340b2de01848d79219a9305b651a9e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43413 } }
I20260811 18:47:49.875674 10250 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:47:49.875731 10250 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:47:49.875798 10250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:49.877154 10250 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86340b2de01848d79219a9305b651a9e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43413 } }
I20260811 18:47:49.877362 10250 leader_election.cc:302] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86340b2de01848d79219a9305b651a9e; no voters:
I20260811 18:47:49.877725 10250 leader_election.cc:288] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:47:49.877869 10262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 1 FOLLOWER]: Leader election won for term 1
Aug 11 18:47:49 dist-test-slave-pww0 krb5kdc[10088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:49.878201 10262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [term 1 LEADER]: Becoming Leader. State: Replica: 86340b2de01848d79219a9305b651a9e, State: Running, Role: LEADER
I20260811 18:47:49.878333 10262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86340b2de01848d79219a9305b651a9e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43413 } }
I20260811 18:47:49.878348 10250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:47:49.879024 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86340b2de01848d79219a9305b651a9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86340b2de01848d79219a9305b651a9e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43413 } } }
I20260811 18:47:49.879122 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [sys.catalog]: This master's current role is: LEADER
I20260811 18:47:49.879578 10267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86340b2de01848d79219a9305b651a9e. Latest consensus state: current_term: 1 leader_uuid: "86340b2de01848d79219a9305b651a9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86340b2de01848d79219a9305b651a9e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43413 } } }
I20260811 18:47:49.879662 10267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e [sys.catalog]: This master's current role is: LEADER
I20260811 18:47:49.880028 10271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:47:49.882602 10271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:47:49.884671 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:49.862118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.9.215.62:43413)
0811 18:47:49.862541 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:49.862550 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:49.862792 (+ 242us) server_negotiation.cc:407] Connection header received
0811 18:47:49.867676 (+ 4884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:49.867685 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:49.867776 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:49.867905 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:49.870470 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:49.871383 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:49.873045 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:49.873449 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:49.873531 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:49.878974 (+ 5443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:49.878979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:49.878980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:49.878995 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:49.882056 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:49.882677 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:49.882681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:49.882682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:49.882818 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:49.883210 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:49.883211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:49.883212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:49.883596 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:49.883845 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:49.884119 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:47:49.884289 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":200,"threads_started":1}
I20260811 18:47:49.888448 10271 catalog_manager.cc:1347] Generated new cluster ID: 1bf2d979b5154172bb693cf1320e9d85
I20260811 18:47:49.888530 10271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:47:49.901588 10271 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:47:49.902222 10271 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:47:49.906869 10271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86340b2de01848d79219a9305b651a9e: Generated new TSK 0
I20260811 18:47:49.907126 10271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:47:49.943255 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43413
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:50.066854 10299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:50.067103 10299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:50.067133 10299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:50.068603 10299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:50.068674 10299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:50.068706 10299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:47:50.070746 10299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:50.071055 10299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:50.071338 10299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:50.079756 10317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:50.079830 10315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:50.080010 10323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:50.080040 10299 server_base.cc:1034] running on GCE node
I20260811 18:47:50.080461 10299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:50.080775 10299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:50.081921 10299 hybrid_clock.cc:648] HybridClock initialized: now 1786474070081904 us; error 36 us; skew 500 ppm
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:50.086046 10299 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:47:50.086567 10299 webserver.cc:460] Webserver started at http://127.9.215.1:42529/ using document root <none> and password file <none>
I20260811 18:47:50.086789 10299 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:50.086841 10299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:50.086977 10299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:50.088404 10299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "d0c18f13e9384b418eea0e0451f5ca1c"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "73da38cca4c9af13bb4bb4bfa1f8e6b2"
server_key_iv: "d67fc5ce9c743e3fa9e8114b29962d5e"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.088819 10299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "d0c18f13e9384b418eea0e0451f5ca1c"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "73da38cca4c9af13bb4bb4bfa1f8e6b2"
server_key_iv: "d67fc5ce9c743e3fa9e8114b29962d5e"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.090453 10299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:47:50.091500 10336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.091867 10299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:50.091981 10299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "d0c18f13e9384b418eea0e0451f5ca1c"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "73da38cca4c9af13bb4bb4bfa1f8e6b2"
server_key_iv: "d67fc5ce9c743e3fa9e8114b29962d5e"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.092056 10299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:50.105406 10299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:50.108880 10299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:50.109107 10299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:50.109549 10299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:47:50.110050 10299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:50.110114 10299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.110147 10299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:50.110172 10299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.127574 10299 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:45355
I20260811 18:47:50.127715 10561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:45355 every 8 connection(s)
I20260811 18:47:50.128062 10299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:47:50.130745 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10299
I20260811 18:47:50.130843 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:47:50.130983 10076 external_mini_cluster.cc:1428] Setting key 59f012e68ee3853991619e958bd2cc98
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:50.141412 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.129135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:53285 (local address 127.9.215.62:43413)
0811 18:47:50.129180 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:50.129182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:50.129562 (+ 380us) server_negotiation.cc:407] Connection header received
0811 18:47:50.130485 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:50.130487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:50.130501 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:50.130555 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:50.132102 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.132624 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.134206 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.134422 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.134484 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.137724 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:50.137726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:50.137727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:50.137738 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:50.139664 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.140216 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.140218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.140218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.140297 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.140569 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.140570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.140571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.140746 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:50.140822 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:50.141159 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 18:47:50.141274 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:47:50.141975 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.129117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43413 (local address 127.9.215.1:53285)
0811 18:47:50.129397 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:50.129439 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:50.130367 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:50.130650 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:50.130654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:50.131633 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:50.132003 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:50.132010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.132716 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.132717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:50.134105 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:50.134109 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.134511 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.134512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:50.134733 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.135499 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:50.135510 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:50.137576 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:50.139801 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:50.139815 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:50.139818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:50.140122 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:50.140373 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:50.140374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:50.140374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:50.140503 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:50.140841 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:50.140851 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:50.141009 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 18:47:50.141113 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:47:50.141441 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:47:50.142853 10562 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43413
I20260811 18:47:50.142980 10562 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:50.143177 10562 heartbeater.cc:511] Master 127.9.215.62:43413 requested a full tablet report, sending...
I20260811 18:47:50.144289 10175 ts_manager.cc:194] Registered new tserver with Master: d0c18f13e9384b418eea0e0451f5ca1c (127.9.215.1:45355)
I20260811 18:47:50.145303 10175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:53285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:47:50.163156 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43413
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:50.297179 10569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:50.297446 10569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:50.297477 10569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:50.298926 10569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:50.298986 10569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:50.299019 10569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:47:50.301420 10569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:50.301743 10569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:50.302011 10569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:50.309218 10578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:50.309244 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:50.309381 10569 server_base.cc:1034] running on GCE node
W20260811 18:47:50.309258 10584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:50.309721 10569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:50.309994 10569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:50.311148 10569 hybrid_clock.cc:648] HybridClock initialized: now 1786474070311124 us; error 43 us; skew 500 ppm
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:50.314831 10569 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:47:50.315426 10569 webserver.cc:460] Webserver started at http://127.9.215.2:44545/ using document root <none> and password file <none>
I20260811 18:47:50.315696 10569 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:50.315752 10569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:50.315887 10569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:50.317188 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "a7fa39ebc41647da99b601f8a286e7b1"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "133eb950114b339245802c68096449b6"
server_key_iv: "2a8169da66b9f52f92c79dbae6c9246a"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.317607 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "a7fa39ebc41647da99b601f8a286e7b1"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "133eb950114b339245802c68096449b6"
server_key_iv: "2a8169da66b9f52f92c79dbae6c9246a"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.319177 10569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:47:50.320254 10600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.320439 10569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:50.320544 10569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "a7fa39ebc41647da99b601f8a286e7b1"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "133eb950114b339245802c68096449b6"
server_key_iv: "2a8169da66b9f52f92c79dbae6c9246a"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.320613 10569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:50.329380 10569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:50.332957 10569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:50.333168 10569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:50.333685 10569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:47:50.334173 10569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:50.334224 10569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.334259 10569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:50.334282 10569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.352010 10569 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:34509
I20260811 18:47:50.352131 10832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:34509 every 8 connection(s)
I20260811 18:47:50.352535 10569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:47:50.360512 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10569
I20260811 18:47:50.360606 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:47:50.360790 10076 external_mini_cluster.cc:1428] Setting key 3914937a3b6119b86faa0642234e639c
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:50.365890 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.353529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:46327 (local address 127.9.215.62:43413)
0811 18:47:50.353603 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:50.353605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:50.354069 (+ 464us) server_negotiation.cc:407] Connection header received
0811 18:47:50.355022 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:50.355024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:50.355036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:50.355115 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:50.356598 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.357143 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.358550 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.358769 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.358837 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.361979 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:50.361981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:50.361981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:50.361990 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:50.363678 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.364410 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.364411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.364412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.364564 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.364915 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.364916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.364917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.365054 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:50.365226 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:50.365718 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 18:47:50.365817 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:47:50.366154 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.353480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43413 (local address 127.9.215.2:46327)
0811 18:47:50.353884 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:50.353930 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:50.354893 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:50.355271 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:50.355275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:50.356131 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:50.356463 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:50.356471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.357250 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.357251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:50.358423 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:50.358426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.358906 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.358907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:50.359138 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.359880 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:50.359890 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:50.361875 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:50.363776 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:50.363787 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:50.363791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:50.364300 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:50.364652 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:50.364654 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:50.364654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:50.364831 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:50.365176 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:50.365187 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:50.365471 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 18:47:50.365612 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 18:47:50.365895 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":207,"threads_started":1}
I20260811 18:47:50.366756 10833 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43413
I20260811 18:47:50.366876 10833 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:50.367076 10833 heartbeater.cc:511] Master 127.9.215.62:43413 requested a full tablet report, sending...
I20260811 18:47:50.367867 10175 ts_manager.cc:194] Registered new tserver with Master: a7fa39ebc41647da99b601f8a286e7b1 (127.9.215.2:34509)
I20260811 18:47:50.368539 10175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:46327
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:47:50.399766 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43413
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:50.503964 10847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:50.504244 10847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:50.504276 10847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:50.505753 10847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:50.505821 10847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:50.505854 10847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:47:50.507952 10847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43413
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:50.508252 10847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:50.508519 10847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:50.514568 10864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:50.514583 10859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:50.514796 10860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:50.515120 10847 server_base.cc:1034] running on GCE node
I20260811 18:47:50.515316 10847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:50.515599 10847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:50.516764 10847 hybrid_clock.cc:648] HybridClock initialized: now 1786474070516750 us; error 28 us; skew 500 ppm
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:50.519394 10847 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:47:50.519901 10847 webserver.cc:460] Webserver started at http://127.9.215.3:34099/ using document root <none> and password file <none>
I20260811 18:47:50.520135 10847 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:50.520183 10847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:50.520299 10847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:50.521431 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "f772b5633be04a90913fe483846f190d"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "613cf94399e827a0518f9c884a631456"
server_key_iv: "1a89424681e640dcfdf45281401c1d15"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.521775 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "f772b5633be04a90913fe483846f190d"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "613cf94399e827a0518f9c884a631456"
server_key_iv: "1a89424681e640dcfdf45281401c1d15"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.523435 10847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:47:50.524307 10880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.524483 10847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:50.524572 10847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "f772b5633be04a90913fe483846f190d"
format_stamp: "Formatted at 2026-08-11 18:47:50 on dist-test-slave-pww0"
server_key: "613cf94399e827a0518f9c884a631456"
server_key_iv: "1a89424681e640dcfdf45281401c1d15"
server_key_version: "encryptionkey@0"
I20260811 18:47:50.524643 10847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:50.542052 10847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:50.544834 10847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:50.545063 10847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:50.545527 10847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:47:50.545938 10847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:50.545976 10847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.546008 10847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:50.546025 10847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:50.564262 10847 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:35231
I20260811 18:47:50.564396 11093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:35231 every 8 connection(s)
I20260811 18:47:50.564776 10847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:47:50.567020 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10847
I20260811 18:47:50.567126 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:47:50.567281 10076 external_mini_cluster.cc:1428] Setting key 4b16d369b3c20d8a7ba5b6a260493e7c
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:50.577163 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.565937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:57491 (local address 127.9.215.62:43413)
0811 18:47:50.566015 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:50.566018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:50.566377 (+ 359us) server_negotiation.cc:407] Connection header received
0811 18:47:50.567384 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:50.567386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:50.567400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:50.567476 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:50.568912 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.569423 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.570703 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.570903 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.570960 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.573797 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:50.573799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:50.573800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:50.573808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:50.575327 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.575891 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.575893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.575894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.575993 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.576338 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.576339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.576340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.576504 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:50.576572 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:50.576979 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 18:47:50.577065 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:47:50.577615 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.565780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43413 (local address 127.9.215.3:57491)
0811 18:47:50.566201 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:50.566251 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:50.567271 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:50.567567 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:50.567571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:50.568411 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:50.568812 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:50.568819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.569514 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.569515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:50.570605 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:50.570608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.570978 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.570979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:50.571194 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.571878 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:50.571887 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:50.573667 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:50.575425 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:50.575435 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:50.575437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:50.575806 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:50.576072 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:50.576073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:50.576074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:50.576252 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:50.576607 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:50.576637 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:50.576803 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 18:47:50.576895 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:47:50.577046 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":205,"threads_started":1}
I20260811 18:47:50.578259 11094 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43413
I20260811 18:47:50.578363 11094 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:50.578543 11094 heartbeater.cc:511] Master 127.9.215.62:43413 requested a full tablet report, sending...
I20260811 18:47:50.579006 10175 ts_manager.cc:194] Registered new tserver with Master: f772b5633be04a90913fe483846f190d (127.9.215.3:35231)
I20260811 18:47:50.579552 10175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:57491
I20260811 18:47:50.579655 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:47:50.609274 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.599960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.9.215.1:45355)
0811 18:47:50.600142 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:50.600148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:50.600162 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:47:50.600298 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:50.600307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:50.600323 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:50.600403 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:50.601272 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.602089 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.603092 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.603643 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.603708 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.606461 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:50.606473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:50.606473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:50.606482 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:50.608026 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.608388 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.608392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.608392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.608459 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.608702 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.608702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.608703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.608865 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:50.608980 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:50.609059 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 18:47:50.609159 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
W20260811 18:47:50.609822 10390 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53306
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[10088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:47:50.636936 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:50.629605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.9.215.1:45355)
0811 18:47:50.629672 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:50.629674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:50.629781 (+ 107us) server_negotiation.cc:407] Connection header received
0811 18:47:50.629857 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:50.629858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:50.629873 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:50.629922 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:50.630524 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.631018 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.631822 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:50.632034 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:50.632091 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:50.634512 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:50.634513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:50.634514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:50.634522 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:50.635870 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.636215 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.636216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.636216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.636273 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:50.636441 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:50.636442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:50.636442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:50.636546 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:50.636658 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:50.636749 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 18:47:50.636841 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:47:50.637517 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10299
I20260811 18:47:50.656132 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10569
I20260811 18:47:50.726065 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10847
I20260811 18:47:50.739176 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10117
2026-08-11T18:47:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2643 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[11122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[11122](info): set up 2 sockets
Aug 11 18:47:50 dist-test-slave-pww0 krb5kdc[11122](info): commencing operation
krb5kdc: starting...
W20260811 18:47:51.976421 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:47:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:47:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:47:52.106940 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45031
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:45031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:52.193573 11145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:52.193828 11145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:52.193856 11145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:52.195113 11145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:47:52.195255 11145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:52.195317 11145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:52.195331 11145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:47:52.195343 11145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:47:52.197286 11145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:45031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:52.197556 11145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:52.197767 11145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:52.202924 11158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:52.202924 11161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:52.202926 11171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:52.203034 11145 server_base.cc:1034] running on GCE node
I20260811 18:47:52.203481 11145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:52.203796 11145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:52.204978 11145 hybrid_clock.cc:648] HybridClock initialized: now 1786474072204953 us; error 32 us; skew 500 ppm
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:52.207614 11145 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:47:52.208108 11145 webserver.cc:460] Webserver started at http://127.9.215.62:45065/ using document root <none> and password file <none>
I20260811 18:47:52.208323 11145 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:52.208367 11145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:52.208485 11145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:52.209583 11145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "7e2402c746474893aa7f01583ec086e7"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "2365d4ea585a9d044575a7d4cb8f1cb5"
server_key_iv: "97eb23fd6d64ff61f964ecb1e55ce7f3"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.209928 11145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "7e2402c746474893aa7f01583ec086e7"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "2365d4ea585a9d044575a7d4cb8f1cb5"
server_key_iv: "97eb23fd6d64ff61f964ecb1e55ce7f3"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.211347 11145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:47:52.212235 11185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.212435 11145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:52.212538 11145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "7e2402c746474893aa7f01583ec086e7"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "2365d4ea585a9d044575a7d4cb8f1cb5"
server_key_iv: "97eb23fd6d64ff61f964ecb1e55ce7f3"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.212611 11145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:52.226774 11145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:52.229215 11145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:52.229382 11145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:52.233497 11145 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:45031
I20260811 18:47:52.233548 11315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:45031 every 8 connection(s)
I20260811 18:47:52.233918 11145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:47:52.234866 11318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:52.237270 11318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7: Bootstrap starting.
I20260811 18:47:52.237890 11318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:52.238174 11318 log.cc:824] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:52.239033 11318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7: No bootstrap required, opened a new log
I20260811 18:47:52.240779 11318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2402c746474893aa7f01583ec086e7" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45031 } }
I20260811 18:47:52.240935 11318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:52.240958 11318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2402c746474893aa7f01583ec086e7, State: Initialized, Role: FOLLOWER
I20260811 18:47:52.241103 11318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2402c746474893aa7f01583ec086e7" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45031 } }
I20260811 18:47:52.241180 11318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:47:52.241216 11318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:47:52.241266 11318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:52.241884 11318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2402c746474893aa7f01583ec086e7" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45031 } }
I20260811 18:47:52.241989 11318 leader_election.cc:302] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e2402c746474893aa7f01583ec086e7; no voters:
I20260811 18:47:52.242175 11318 leader_election.cc:288] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:47:52.242229 11324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:52.242415 11324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [term 1 LEADER]: Becoming Leader. State: Replica: 7e2402c746474893aa7f01583ec086e7, State: Running, Role: LEADER
I20260811 18:47:52.242465 11318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:47:52.242532 11324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2402c746474893aa7f01583ec086e7" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45031 } }
I20260811 18:47:52.243049 11325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e2402c746474893aa7f01583ec086e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2402c746474893aa7f01583ec086e7" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45031 } } }
I20260811 18:47:52.243126 11325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:47:52.243144 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11145
I20260811 18:47:52.243217 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:47:52.243392 10076 external_mini_cluster.cc:1428] Setting key 094ffec07270b72e6f5f8dfee1a5369f
I20260811 18:47:52.242905 11326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e2402c746474893aa7f01583ec086e7. Latest consensus state: current_term: 1 leader_uuid: "7e2402c746474893aa7f01583ec086e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2402c746474893aa7f01583ec086e7" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45031 } } }
I20260811 18:47:52.243618 11326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7 [sys.catalog]: This master's current role is: LEADER
W20260811 18:47:52.244307 11356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:47:52.244370 11356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:47:52.244519 11341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:47:52.244694 11341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:47:52.246392 11341 catalog_manager.cc:1347] Generated new cluster ID: e282bf959b034dbf945f5c57f4f650e6
I20260811 18:47:52.246459 11341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:52.254693 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.243923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.9.215.62:45031)
0811 18:47:52.244220 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.244231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.244260 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:47:52.244367 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.244375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.244440 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.244580 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:52.245258 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.246563 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.247323 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.247882 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.247986 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.250354 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:52.250359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:52.250359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:52.250372 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:52.253011 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.253416 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.253419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.253420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.253537 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.253734 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.253734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.253735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.254007 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:52.254171 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.254297 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.254459 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":94,"threads_started":1}
I20260811 18:47:52.260322 11341 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:47:52.260970 11341 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:47:52.270678 11341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e2402c746474893aa7f01583ec086e7: Generated new TSK 0
I20260811 18:47:52.270906 11341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:47:52.307070 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45031
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:52.428711 11366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:52.428973 11366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:52.429001 11366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:52.430284 11366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:52.430336 11366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:52.430377 11366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:47:52.432125 11366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:52.432394 11366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:52.432624 11366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:52.438033 11379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:52.438047 11380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:52.438073 11387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:52.438052 11366 server_base.cc:1034] running on GCE node
I20260811 18:47:52.438486 11366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:52.438752 11366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:52.439916 11366 hybrid_clock.cc:648] HybridClock initialized: now 1786474072439897 us; error 37 us; skew 500 ppm
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:52.442765 11366 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:47:52.443320 11366 webserver.cc:460] Webserver started at http://127.9.215.1:36765/ using document root <none> and password file <none>
I20260811 18:47:52.443641 11366 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:52.443698 11366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:52.443836 11366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:52.444991 11366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "3c387e30318945f7a401e731ace3141f"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "8816cb3350159723748c25d9e7930b93"
server_key_iv: "36a9a65bc752428353b2017b8a6149ae"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.445344 11366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "3c387e30318945f7a401e731ace3141f"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "8816cb3350159723748c25d9e7930b93"
server_key_iv: "36a9a65bc752428353b2017b8a6149ae"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.446794 11366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:47:52.447679 11406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.447860 11366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:52.447953 11366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "3c387e30318945f7a401e731ace3141f"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "8816cb3350159723748c25d9e7930b93"
server_key_iv: "36a9a65bc752428353b2017b8a6149ae"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.448032 11366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:52.469158 11366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:52.471982 11366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:52.472148 11366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:52.472415 11366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:47:52.472770 11366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:52.472810 11366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.472841 11366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:52.472858 11366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.478802 11366 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:42445
I20260811 18:47:52.478907 11651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:42445 every 8 connection(s)
I20260811 18:47:52.479202 11366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:47:52.484912 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11366
I20260811 18:47:52.485001 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:47:52.485148 10076 external_mini_cluster.cc:1428] Setting key a23ce1197a3fbd095ea60ff3cdb921b9
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:52.489723 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.480078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:40883 (local address 127.9.215.62:45031)
0811 18:47:52.480143 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.480145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.480350 (+ 205us) server_negotiation.cc:407] Connection header received
0811 18:47:52.481481 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.481483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.481499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.481557 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:52.482245 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.482764 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.483900 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.484117 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.484172 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.486731 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:52.486732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:52.486733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:52.486740 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:52.488346 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.488761 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.488762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.488763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.488822 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.489047 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.489049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.489049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.489172 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:52.489232 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.489527 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.489629 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:47:52.489895 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.479989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45031 (local address 127.9.215.1:40883)
0811 18:47:52.480217 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:52.480261 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:52.481364 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:52.481642 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:52.481646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:52.481934 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:52.482157 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.482162 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.482874 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.482875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.483806 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.483809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.484202 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.484203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.484328 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.484861 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:52.484870 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:52.486617 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:52.488440 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:52.488449 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:52.488451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:52.488689 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:52.488876 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:52.488876 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:52.488877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:52.488974 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:52.489240 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:52.489249 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:52.489399 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 18:47:52.489493 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:47:52.489665 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
I20260811 18:47:52.490417 11653 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45031
I20260811 18:47:52.490540 11653 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:52.490777 11653 heartbeater.cc:511] Master 127.9.215.62:45031 requested a full tablet report, sending...
I20260811 18:47:52.491425 11226 ts_manager.cc:194] Registered new tserver with Master: 3c387e30318945f7a401e731ace3141f (127.9.215.1:42445)
I20260811 18:47:52.492177 11226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:40883
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:47:52.515506 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45031
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:52.601872 11666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:52.602113 11666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:52.602140 11666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:52.603423 11666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:52.603518 11666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:52.603565 11666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:47:52.605274 11666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:52.605566 11666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:52.605813 11666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:52.611204 11678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:52.611204 11675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:52.611289 11666 server_base.cc:1034] running on GCE node
W20260811 18:47:52.611208 11680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:52.611725 11666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:52.612007 11666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:52.613180 11666 hybrid_clock.cc:648] HybridClock initialized: now 1786474072613155 us; error 45 us; skew 500 ppm
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:52.615962 11666 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:47:52.616456 11666 webserver.cc:460] Webserver started at http://127.9.215.2:40739/ using document root <none> and password file <none>
I20260811 18:47:52.616670 11666 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:52.616714 11666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:52.616828 11666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:52.617865 11666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "b95f9a65f20943e4a80dab82f823240d"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "cd0cc9fcb061596dc900114c517e54da"
server_key_iv: "280a32315e9d4abaaaab0484c94ed372"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.618188 11666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "b95f9a65f20943e4a80dab82f823240d"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "cd0cc9fcb061596dc900114c517e54da"
server_key_iv: "280a32315e9d4abaaaab0484c94ed372"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.619524 11666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:47:52.620296 11694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.620450 11666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:47:52.620546 11666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "b95f9a65f20943e4a80dab82f823240d"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "cd0cc9fcb061596dc900114c517e54da"
server_key_iv: "280a32315e9d4abaaaab0484c94ed372"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.620615 11666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:52.647164 11666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:52.649627 11666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:52.649799 11666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:52.650055 11666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:47:52.650390 11666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:52.650427 11666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.650458 11666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:52.650475 11666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.656347 11666 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42219
I20260811 18:47:52.656440 11935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42219 every 8 connection(s)
I20260811 18:47:52.656764 11666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:47:52.661863 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11666
I20260811 18:47:52.661976 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:47:52.662175 10076 external_mini_cluster.cc:1428] Setting key e726e3d69a4b7347e32a3b667b547ef0
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:52.667784 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.657595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:45407 (local address 127.9.215.62:45031)
0811 18:47:52.657669 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.657671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.657899 (+ 228us) server_negotiation.cc:407] Connection header received
0811 18:47:52.658909 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.658910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.658922 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.658989 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:52.659781 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.660318 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.661409 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.661622 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.661688 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.664453 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:52.664455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:52.664455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:52.664463 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:52.665979 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.666601 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.666603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.666603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.666676 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.666989 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.666990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.666990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.667141 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:52.667209 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.667560 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.667667 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:47:52.668009 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.657557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45031 (local address 127.9.215.2:45407)
0811 18:47:52.657776 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:52.657817 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:52.658771 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:52.659070 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:52.659074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:52.659376 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:52.659654 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.659660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.660428 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.660429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.661303 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.661306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.661746 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.661747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.661933 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.662560 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:52.662571 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:52.664304 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:52.666094 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:52.666131 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:52.666134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:52.666463 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:52.666774 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:52.666776 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:52.666776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:52.666917 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:52.667229 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:52.667238 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:52.667390 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 18:47:52.667535 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 18:47:52.667770 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260811 18:47:52.668617 11936 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45031
I20260811 18:47:52.668717 11936 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:52.668905 11936 heartbeater.cc:511] Master 127.9.215.62:45031 requested a full tablet report, sending...
I20260811 18:47:52.669318 11226 ts_manager.cc:194] Registered new tserver with Master: b95f9a65f20943e4a80dab82f823240d (127.9.215.2:42219)
I20260811 18:47:52.669685 11226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:45407
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:47:52.691604 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45031
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:52.779572 11946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:52.779806 11946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:52.779834 11946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:52.781005 11946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:52.781044 11946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:52.781072 11946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:47:52.782589 11946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:52.782841 11946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:52.783059 11946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:52.788568 11953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:52.788576 11955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:52.788787 11946 server_base.cc:1034] running on GCE node
W20260811 18:47:52.788576 11959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:52.789093 11946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:52.789366 11946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:52.790580 11946 hybrid_clock.cc:648] HybridClock initialized: now 1786474072790533 us; error 54 us; skew 500 ppm
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:52.793222 11946 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:47:52.793723 11946 webserver.cc:460] Webserver started at http://127.9.215.3:34511/ using document root <none> and password file <none>
I20260811 18:47:52.793921 11946 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:52.793963 11946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:52.794054 11946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:52.795095 11946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "02948c0f2f1d4e279225dec9ffa1af01"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "d4f218cdb6ab79201a4db4c17130c9a6"
server_key_iv: "8a59fe3e2012e8ff14d5b91a0ac355df"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.795410 11946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "02948c0f2f1d4e279225dec9ffa1af01"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "d4f218cdb6ab79201a4db4c17130c9a6"
server_key_iv: "8a59fe3e2012e8ff14d5b91a0ac355df"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.796792 11946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:52.797564 11972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.797780 11946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:52.797868 11946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "02948c0f2f1d4e279225dec9ffa1af01"
format_stamp: "Formatted at 2026-08-11 18:47:52 on dist-test-slave-pww0"
server_key: "d4f218cdb6ab79201a4db4c17130c9a6"
server_key_iv: "8a59fe3e2012e8ff14d5b91a0ac355df"
server_key_version: "encryptionkey@0"
I20260811 18:47:52.797930 11946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:52.813153 11946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:52.815837 11946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:52.815975 11946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:52.816182 11946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:47:52.816534 11946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:52.816582 11946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.816607 11946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:52.816632 11946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:52.822569 11946 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:35057
I20260811 18:47:52.822619 12205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:35057 every 8 connection(s)
I20260811 18:47:52.822968 11946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:47:52.828006 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11946
I20260811 18:47:52.828104 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:47:52.828266 10076 external_mini_cluster.cc:1428] Setting key fed832e79c81530a30679eeb5b1ae38c
Aug 11 18:47:52 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:52.832887 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.823778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:58271 (local address 127.9.215.62:45031)
0811 18:47:52.823845 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.823847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.824088 (+ 241us) server_negotiation.cc:407] Connection header received
0811 18:47:52.825153 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.825155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.825170 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.825224 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:52.825920 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.826431 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.827508 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.827711 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.827771 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.830236 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:52.830238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:52.830238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:52.830254 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:52.831488 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.831900 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.831902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.831902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.831964 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.832204 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.832205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.832205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.832351 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:52.832412 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.832697 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.832791 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:47:52.833094 12210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.823785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45031 (local address 127.9.215.3:58271)
0811 18:47:52.823958 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:52.823999 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:52.825041 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:52.825302 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:52.825306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:52.825599 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:52.825820 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.825825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.826528 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.826529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.827380 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.827382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.827800 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.827801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.827935 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.828575 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:52.828587 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:52.830142 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:52.831572 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:52.831583 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:52.831586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:52.831830 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:52.832021 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:52.832022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:52.832023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:52.832146 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:52.832409 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:52.832418 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:52.832572 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 18:47:52.832667 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:47:52.832856 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260811 18:47:52.833578 12206 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45031
I20260811 18:47:52.833675 12206 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:52.833871 12206 heartbeater.cc:511] Master 127.9.215.62:45031 requested a full tablet report, sending...
I20260811 18:47:52.834299 11226 ts_manager.cc:194] Registered new tserver with Master: 02948c0f2f1d4e279225dec9ffa1af01 (127.9.215.3:35057)
I20260811 18:47:52.834700 11226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:58271
I20260811 18:47:52.840373 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:47:52.850289 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.842593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.9.215.62:45031)
0811 18:47:52.842686 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.842688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.842789 (+ 101us) server_negotiation.cc:407] Connection header received
0811 18:47:52.842863 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.842864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.842876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.842926 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:52.843627 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.844167 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.845489 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.845860 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.845949 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.847282 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:52.847284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:52.847285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:52.847293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:52.849043 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.849446 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.849447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.849448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.849508 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:52.849683 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:52.849684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:52.849685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:52.849858 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:52.849952 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.850080 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.850176 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:47:52.853629 11226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:47:52.855581 11226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:52.863561 12227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.858846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:35057 (local address 127.0.0.1:58154)
0811 18:47:52.859033 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:52.859076 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:52.859143 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:52.859562 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:52.859565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:52.859567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:52.859651 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.859653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.860971 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.860972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.862354 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.862358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.863204 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.863204 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.863285 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.863299 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 18:47:52.863373 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:47:52.863436 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":74,"thread_start_us":83,"threads_started":1}
I20260811 18:47:52.863648 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.858846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42445 (local address 127.0.0.1:44138)
0811 18:47:52.859180 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:52.859191 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:52.859240 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:52.859854 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:52.859855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:52.859856 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:52.859989 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.859991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.861371 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.861372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.862277 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.862280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.863339 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.863340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.863405 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.863409 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:52.863488 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:47:52.863524 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260811 18:47:52.863659 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.858957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.9.215.1:42445)
0811 18:47:52.859702 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.859706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.859716 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:47:52.859754 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.859756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.859770 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.859855 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:52.860085 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.861203 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.862354 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.863258 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.863325 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.863381 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.863493 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.863535 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":93,"threads_started":1}
I20260811 18:47:52.864120 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.858945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.9.215.3:35057)
0811 18:47:52.859367 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.859372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.859381 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:47:52.859418 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.859420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.859436 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.859540 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:52.859743 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.860865 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.862430 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.863121 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.863206 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.863281 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.863971 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.864021 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":101,"threads_started":1}
I20260811 18:47:52.864697 11505 tablet_service.cc:1467] Processing CreateTablet for tablet 666dedc1595146c8bd9504110d83f6b9 (DEFAULT_TABLE table=test-table [id=b619ca1828494dfcbf3fc37c091c2ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:52.864836 12061 tablet_service.cc:1467] Processing CreateTablet for tablet 666dedc1595146c8bd9504110d83f6b9 (DEFAULT_TABLE table=test-table [id=b619ca1828494dfcbf3fc37c091c2ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:52.865175 12061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666dedc1595146c8bd9504110d83f6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:52.865200 12225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.859057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42219 (local address 127.0.0.1:34392)
0811 18:47:52.859248 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:52.859256 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:52.859296 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:52.859681 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:52.859681 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:52.859683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:52.859726 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.859728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.860918 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.860920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.862364 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:52.862366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.864942 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.864943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:52.865010 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.865015 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:47:52.865081 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:47:52.865129 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":78,"thread_start_us":80,"threads_started":1}
I20260811 18:47:52.865170 11505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666dedc1595146c8bd9504110d83f6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:52.865257 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:52.859341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.9.215.2:42219)
0811 18:47:52.859528 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:52.859533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:52.859541 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:47:52.859580 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:52.859582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:52.859596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:52.859675 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:52.859788 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.860705 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.863326 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:52.864842 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:52.864922 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:52.864983 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:52.865099 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 18:47:52.865143 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":76,"threads_started":1}
I20260811 18:47:52.865983 11804 tablet_service.cc:1467] Processing CreateTablet for tablet 666dedc1595146c8bd9504110d83f6b9 (DEFAULT_TABLE table=test-table [id=b619ca1828494dfcbf3fc37c091c2ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:52.866310 11804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666dedc1595146c8bd9504110d83f6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:52.869722 12237 tablet_bootstrap.cc:492] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: Bootstrap starting.
I20260811 18:47:52.870607 12237 tablet_bootstrap.cc:654] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:52.870958 12240 tablet_bootstrap.cc:492] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: Bootstrap starting.
I20260811 18:47:52.871671 12239 tablet_bootstrap.cc:492] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: Bootstrap starting.
I20260811 18:47:52.871994 12237 log.cc:824] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:52.872628 12240 tablet_bootstrap.cc:654] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:52.873164 12237 tablet_bootstrap.cc:492] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: No bootstrap required, opened a new log
I20260811 18:47:52.873247 12237 ts_tablet_manager.cc:1397] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:47:52.873346 12239 tablet_bootstrap.cc:654] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:52.874267 12239 log.cc:824] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:52.873968 12240 log.cc:824] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:52.875250 12239 tablet_bootstrap.cc:492] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: No bootstrap required, opened a new log
I20260811 18:47:52.875304 12240 tablet_bootstrap.cc:492] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: No bootstrap required, opened a new log
I20260811 18:47:52.875336 12239 ts_tablet_manager.cc:1397] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:47:52.875367 12240 ts_tablet_manager.cc:1397] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 18:47:52.877733 12237 raft_consensus.cc:348] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:52.878068 12237 raft_consensus.cc:374] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:52.878134 12237 raft_consensus.cc:729] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c387e30318945f7a401e731ace3141f, State: Initialized, Role: FOLLOWER
I20260811 18:47:52.878371 12237 consensus_queue.cc:260] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:52.878554 12240 raft_consensus.cc:348] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:52.878834 12240 raft_consensus.cc:374] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:52.878888 12240 raft_consensus.cc:729] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b95f9a65f20943e4a80dab82f823240d, State: Initialized, Role: FOLLOWER
I20260811 18:47:52.878916 12237 ts_tablet_manager.cc:1428] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260811 18:47:52.879019 12240 consensus_queue.cc:260] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:52.879093 11653 heartbeater.cc:503] Master 127.9.215.62:45031 was elected leader, sending a full tablet report...
I20260811 18:47:52.879511 11936 heartbeater.cc:503] Master 127.9.215.62:45031 was elected leader, sending a full tablet report...
I20260811 18:47:52.879524 12240 ts_tablet_manager.cc:1428] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 18:47:52.879719 12239 raft_consensus.cc:348] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:52.879894 12239 raft_consensus.cc:374] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:52.879930 12239 raft_consensus.cc:729] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02948c0f2f1d4e279225dec9ffa1af01, State: Initialized, Role: FOLLOWER
I20260811 18:47:52.880061 12239 consensus_queue.cc:260] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:52.880559 12206 heartbeater.cc:503] Master 127.9.215.62:45031 was elected leader, sending a full tablet report...
I20260811 18:47:52.880784 12239 ts_tablet_manager.cc:1428] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
W20260811 18:47:52.907537 11937 tablet.cc:2369] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:47:52.980088 11654 tablet.cc:2369] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:47:53.073665 12209 tablet.cc:2369] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:47:53.224246 12256 raft_consensus.cc:482] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:53.224390 12256 raft_consensus.cc:504] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:53.225162 12256 leader_election.cc:288] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c387e30318945f7a401e731ace3141f (127.9.215.1:42445), b95f9a65f20943e4a80dab82f823240d (127.9.215.2:42219)
I20260811 18:47:53.228832 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.225449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:49435 (local address 127.9.215.1:42445)
0811 18:47:53.225532 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:53.225534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:53.225566 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:47:53.225659 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:53.225660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:53.225674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:53.225740 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:53.226050 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.226669 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.227927 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.228486 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.228571 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.228583 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:53.228723 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 18:47:53.228767 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:47:53.228827 12210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.225444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42445 (local address 127.9.215.3:49435)
0811 18:47:53.225503 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:53.225516 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:53.225583 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:53.225752 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:53.225753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:53.225755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:53.225953 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:53.225955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.226764 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.226765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:53.227817 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:53.227820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.228564 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.228565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:53.228656 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.228660 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:53.228696 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 18:47:53.228738 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260811 18:47:53.229349 11548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666dedc1595146c8bd9504110d83f6b9" candidate_uuid: "02948c0f2f1d4e279225dec9ffa1af01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c387e30318945f7a401e731ace3141f" is_pre_election: true
I20260811 18:47:53.229604 11548 raft_consensus.cc:2454] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02948c0f2f1d4e279225dec9ffa1af01 in term 0.
I20260811 18:47:53.229808 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.225690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:43491 (local address 127.9.215.2:42219)
0811 18:47:53.225741 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:53.225743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:53.225981 (+ 238us) server_negotiation.cc:407] Connection header received
0811 18:47:53.226013 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:53.226014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:53.226028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:53.226079 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:53.226412 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.227003 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.228765 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.229268 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.229327 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.229337 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:53.229672 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 18:47:53.229718 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:47:53.229817 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.225590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42219 (local address 127.9.215.3:43491)
0811 18:47:53.225912 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:53.225921 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:53.225975 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:53.226096 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:53.226097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:53.226099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:53.226314 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:53.226316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.227113 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.227114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:53.228671 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:53.228674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.229459 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.229461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:53.229590 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.229595 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:47:53.229663 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:47:53.229706 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":206,"threads_started":1}
I20260811 18:47:53.229943 11973 leader_election.cc:302] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02948c0f2f1d4e279225dec9ffa1af01, 3c387e30318945f7a401e731ace3141f; no voters:
I20260811 18:47:53.230067 12256 raft_consensus.cc:2781] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:53.230140 12256 raft_consensus.cc:482] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:53.230165 12256 raft_consensus.cc:3037] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:53.230333 11842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666dedc1595146c8bd9504110d83f6b9" candidate_uuid: "02948c0f2f1d4e279225dec9ffa1af01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b95f9a65f20943e4a80dab82f823240d" is_pre_election: true
I20260811 18:47:53.230472 11842 raft_consensus.cc:2454] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02948c0f2f1d4e279225dec9ffa1af01 in term 0.
I20260811 18:47:53.231391 12256 raft_consensus.cc:504] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:53.231554 12256 leader_election.cc:288] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [CANDIDATE]: Term 1 election: Requested vote from peers 3c387e30318945f7a401e731ace3141f (127.9.215.1:42445), b95f9a65f20943e4a80dab82f823240d (127.9.215.2:42219)
I20260811 18:47:53.231832 11548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666dedc1595146c8bd9504110d83f6b9" candidate_uuid: "02948c0f2f1d4e279225dec9ffa1af01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c387e30318945f7a401e731ace3141f"
I20260811 18:47:53.231830 11842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666dedc1595146c8bd9504110d83f6b9" candidate_uuid: "02948c0f2f1d4e279225dec9ffa1af01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b95f9a65f20943e4a80dab82f823240d"
I20260811 18:47:53.231932 11548 raft_consensus.cc:3037] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:53.231932 11842 raft_consensus.cc:3037] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:53.233096 11842 raft_consensus.cc:2454] T 666dedc1595146c8bd9504110d83f6b9 P b95f9a65f20943e4a80dab82f823240d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02948c0f2f1d4e279225dec9ffa1af01 in term 1.
I20260811 18:47:53.233096 11548 raft_consensus.cc:2454] T 666dedc1595146c8bd9504110d83f6b9 P 3c387e30318945f7a401e731ace3141f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02948c0f2f1d4e279225dec9ffa1af01 in term 1.
I20260811 18:47:53.233305 11973 leader_election.cc:302] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02948c0f2f1d4e279225dec9ffa1af01, b95f9a65f20943e4a80dab82f823240d; no voters:
I20260811 18:47:53.233413 12256 raft_consensus.cc:2781] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:53.233574 12256 raft_consensus.cc:686] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [term 1 LEADER]: Becoming Leader. State: Replica: 02948c0f2f1d4e279225dec9ffa1af01, State: Running, Role: LEADER
I20260811 18:47:53.233686 12256 consensus_queue.cc:237] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } }
I20260811 18:47:53.234540 11226 catalog_manager.cc:5495] T 666dedc1595146c8bd9504110d83f6b9 P 02948c0f2f1d4e279225dec9ffa1af01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02948c0f2f1d4e279225dec9ffa1af01 (127.9.215.3). New cstate: current_term: 1 leader_uuid: "02948c0f2f1d4e279225dec9ffa1af01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02948c0f2f1d4e279225dec9ffa1af01" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 35057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c387e30318945f7a401e731ace3141f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b95f9a65f20943e4a80dab82f823240d" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42219 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:47:53.320475 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.304774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.9.215.1:42445)
0811 18:47:53.304843 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:53.304845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:53.305027 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:47:53.305104 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:53.305105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:53.305116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:53.305166 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:53.306112 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.306699 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.307847 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.308164 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.308220 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.316965 (+ 8745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:53.316988 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:53.316988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:53.317000 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:53.318888 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:53.319400 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:53.319405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:53.319405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:53.319532 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:53.319801 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:53.319802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:53.319803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:53.320023 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:53.320170 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:53.320286 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 18:47:53.320375 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474073, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474073, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:47:53.347064 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.339548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.9.215.1:42445)
0811 18:47:53.339647 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:53.339649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:53.339755 (+ 106us) server_negotiation.cc:407] Connection header received
0811 18:47:53.339833 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:53.339834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:53.339845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:53.339895 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:53.340483 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.341073 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.341847 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.342060 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.342117 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.344492 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:53.344494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:53.344494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:53.344503 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:53.345843 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:53.346256 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:53.346257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:53.346257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:53.346328 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:53.346533 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:53.346535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:53.346535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:53.346659 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:53.346772 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:53.346915 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:47:53.346992 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[11122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[11122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:47:53.372465 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:53.365244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.9.215.1:42445)
0811 18:47:53.365331 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:53.365333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:53.365410 (+ 77us) server_negotiation.cc:407] Connection header received
0811 18:47:53.365488 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:53.365489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:53.365500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:53.365551 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:53.366139 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.366631 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.367374 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:53.367676 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:53.367745 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:53.370082 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:53.370084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:53.370084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:53.370093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:53.371374 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:53.371730 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:53.371732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:53.371732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:53.371792 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:53.371988 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:53.371989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:53.371990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:53.372098 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:53.372187 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:53.372285 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:47:53.372366 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 18:47:53.373116 11461 tablet_service.cc:2908] Rejecting scan request for tablet 666dedc1595146c8bd9504110d83f6b9: Uninitialized: clean time has not yet been initialized
I20260811 18:47:53.373955 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11366
I20260811 18:47:53.394418 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11666
I20260811 18:47:53.436070 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11946
I20260811 18:47:53.451601 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11145
2026-08-11T18:47:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2724 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[12296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[12296](info): set up 2 sockets
Aug 11 18:47:53 dist-test-slave-pww0 krb5kdc[12296](info): commencing operation
krb5kdc: starting...
W20260811 18:47:54.690990 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:47:54 dist-test-slave-pww0 krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:47:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:47:54.819321 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:42413
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:42413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:54.904703 12327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:54.904932 12327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:54.904965 12327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:54.906213 12327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:47:54.906267 12327 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:47:54.906284 12327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:54.906301 12327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:54.906317 12327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:47:54.906332 12327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:47:54.907994 12327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:42413
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:42413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:54.908286 12327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:54.908511 12327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:54.913708 12338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:54.913726 12334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:54.913728 12335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:54.913913 12327 server_base.cc:1034] running on GCE node
I20260811 18:47:54.914222 12327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:54.914547 12327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:54.915699 12327 hybrid_clock.cc:648] HybridClock initialized: now 1786474074915679 us; error 31 us; skew 500 ppm
Aug 11 18:47:54 dist-test-slave-pww0 krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474074, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:54.918301 12327 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:47:54.918816 12327 webserver.cc:460] Webserver started at http://127.9.215.62:40083/ using document root <none> and password file <none>
I20260811 18:47:54.919018 12327 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:54.919060 12327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:54.919162 12327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:54.920316 12327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "d9f396f37b6d4ec2a284ce726612a69a"
format_stamp: "Formatted at 2026-08-11 18:47:54 on dist-test-slave-pww0"
server_key: "34e188960af8ccd2fe85f6f288a6e8d7"
server_key_iv: "413f4b55d0f7af7a97c7b80c68038d0f"
server_key_version: "encryptionkey@0"
I20260811 18:47:54.920657 12327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "d9f396f37b6d4ec2a284ce726612a69a"
format_stamp: "Formatted at 2026-08-11 18:47:54 on dist-test-slave-pww0"
server_key: "34e188960af8ccd2fe85f6f288a6e8d7"
server_key_iv: "413f4b55d0f7af7a97c7b80c68038d0f"
server_key_version: "encryptionkey@0"
I20260811 18:47:54.922096 12327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:47:54.923100 12350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:54.923312 12327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:47:54.923414 12327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "d9f396f37b6d4ec2a284ce726612a69a"
format_stamp: "Formatted at 2026-08-11 18:47:54 on dist-test-slave-pww0"
server_key: "34e188960af8ccd2fe85f6f288a6e8d7"
server_key_iv: "413f4b55d0f7af7a97c7b80c68038d0f"
server_key_version: "encryptionkey@0"
I20260811 18:47:54.923522 12327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:54.939158 12327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:54.941814 12327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:54.941957 12327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:54.947873 12327 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:42413
I20260811 18:47:54.947948 12467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:42413 every 8 connection(s)
I20260811 18:47:54.948274 12327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:47:54.949019 12470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:54.951601 12470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: Bootstrap starting.
I20260811 18:47:54.952241 12470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:54.952545 12470 log.cc:824] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:54.953410 12470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: No bootstrap required, opened a new log
I20260811 18:47:54.955617 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12327
I20260811 18:47:54.955708 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:47:54.955881 10076 external_mini_cluster.cc:1428] Setting key 1ecba2bc20d2e6f8d4afdcd8a28cc2fd
I20260811 18:47:54.956032 12470 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f396f37b6d4ec2a284ce726612a69a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42413 } }
I20260811 18:47:54.956238 12470 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:54.956284 12470 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9f396f37b6d4ec2a284ce726612a69a, State: Initialized, Role: FOLLOWER
I20260811 18:47:54.956439 12470 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f396f37b6d4ec2a284ce726612a69a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42413 } }
I20260811 18:47:54.956535 12470 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:47:54.956580 12470 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:47:54.956625 12470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:54.957440 12470 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f396f37b6d4ec2a284ce726612a69a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42413 } }
I20260811 18:47:54.957540 12470 leader_election.cc:302] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9f396f37b6d4ec2a284ce726612a69a; no voters:
I20260811 18:47:54.957739 12470 leader_election.cc:288] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:47:54.957782 12480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:54.957906 12480 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [term 1 LEADER]: Becoming Leader. State: Replica: d9f396f37b6d4ec2a284ce726612a69a, State: Running, Role: LEADER
I20260811 18:47:54.957994 12480 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f396f37b6d4ec2a284ce726612a69a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42413 } }
I20260811 18:47:54.958036 12470 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:47:54.959369 12486 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9f396f37b6d4ec2a284ce726612a69a. Latest consensus state: current_term: 1 leader_uuid: "d9f396f37b6d4ec2a284ce726612a69a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f396f37b6d4ec2a284ce726612a69a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42413 } } }
I20260811 18:47:54.959512 12486 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [sys.catalog]: This master's current role is: LEADER
I20260811 18:47:54.959893 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9f396f37b6d4ec2a284ce726612a69a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f396f37b6d4ec2a284ce726612a69a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42413 } } }
I20260811 18:47:54.959966 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a [sys.catalog]: This master's current role is: LEADER
W20260811 18:47:54.960134 12508 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:47:54.960196 12508 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:47:54.960247 12511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:47:54.960428 12511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:47:54.962208 12511 catalog_manager.cc:1347] Generated new cluster ID: b4e1ee4d72aa4b659d308b6a9de13bf6
I20260811 18:47:54.962266 12511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:47:54 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:54.967252 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:54.956515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.9.215.62:42413)
0811 18:47:54.956729 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:54.956740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:54.956776 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:47:54.956928 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:54.956936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:54.957018 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:54.957172 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:54.957874 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:54.959249 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:54.960387 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:54.960990 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:54.961122 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:54.963666 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:54.963670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:54.963671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:54.963685 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:54.965589 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:54.965985 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:54.965987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:54.965988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:54.966100 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:54.966309 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:54.966309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:54.966309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:54.966611 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:54.966749 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:54.966886 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:47:54.967027 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
I20260811 18:47:54.979525 12511 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:47:54.980317 12511 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:47:54.987700 12511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: Generated new TSK 0
I20260811 18:47:54.987944 12511 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:47:55.027578 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:42413
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:55.147950 12520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:55.148188 12520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:55.148217 12520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:55.149503 12520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:55.149554 12520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:55.149575 12520 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:47:55.149601 12520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:47:55.151187 12520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:42413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:55.151429 12520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:55.151695 12520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:55.157784 12528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:55.157763 12527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:55.157826 12520 server_base.cc:1034] running on GCE node
W20260811 18:47:55.157792 12534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:55.158512 12520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:55.158826 12520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:55.160010 12520 hybrid_clock.cc:648] HybridClock initialized: now 1786474075159994 us; error 33 us; skew 500 ppm
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:55.162724 12520 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:47:55.163210 12520 webserver.cc:460] Webserver started at http://127.9.215.1:35503/ using document root <none> and password file <none>
I20260811 18:47:55.163450 12520 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:55.163537 12520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:55.163676 12520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:55.164790 12520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "a1a97a06fa504ca1a9a6f4376617062e"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "6eea6b5ea6989b5ac89f9353b3b8ddb0"
server_key_iv: "4fc2d2b983aa75c4adbe53e0c3c3e287"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.165196 12520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "a1a97a06fa504ca1a9a6f4376617062e"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "6eea6b5ea6989b5ac89f9353b3b8ddb0"
server_key_iv: "4fc2d2b983aa75c4adbe53e0c3c3e287"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.166611 12520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:47:55.167549 12549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.167780 12520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:55.167887 12520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "a1a97a06fa504ca1a9a6f4376617062e"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "6eea6b5ea6989b5ac89f9353b3b8ddb0"
server_key_iv: "4fc2d2b983aa75c4adbe53e0c3c3e287"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.167960 12520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:55.178494 12520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:55.180892 12520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:55.181048 12520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:55.181594 12520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:55.181650 12520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.181684 12520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:55.181701 12520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.188726 12520 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:38663
I20260811 18:47:55.188850 12785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:38663 every 8 connection(s)
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:55.189154 12520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:47:55.192391 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.182298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:42045 (local address 127.9.215.62:42413)
0811 18:47:55.182352 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.182353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.182585 (+ 232us) server_negotiation.cc:407] Connection header received
0811 18:47:55.183164 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.183166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.183186 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.183251 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.184270 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.185062 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.186649 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.186940 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.186998 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.189649 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.189651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.189651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.189662 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.191018 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.191517 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.191519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.191519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.191596 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.191832 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.191833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.191833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.191949 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.191999 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.192225 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.192321 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:47:55.192548 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.182268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.9.215.1:42045)
0811 18:47:55.182453 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.182493 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.183046 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.183388 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.183394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.183818 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.184146 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.184154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.185173 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.185174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.186545 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.186549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.187029 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.187031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.187217 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.187902 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.187914 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.189543 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.191125 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.191134 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.191136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.191373 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.191671 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.191671 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.191672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.191787 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.191987 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.191990 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.192131 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.192207 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.192343 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260811 18:47:55.193121 12786 heartbeater.cc:347] Connected to a master server at 127.9.215.62:42413
I20260811 18:47:55.193228 12786 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:55.193445 12786 heartbeater.cc:511] Master 127.9.215.62:42413 requested a full tablet report, sending...
I20260811 18:47:55.193996 12385 ts_manager.cc:194] Registered new tserver with Master: a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1:38663)
I20260811 18:47:55.194751 12385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:42045
I20260811 18:47:55.195601 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12520
I20260811 18:47:55.195694 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:47:55.195863 10076 external_mini_cluster.cc:1428] Setting key 44c041748cb2b170e2b5b9799992f79a
I20260811 18:47:55.200250 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.194349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:59791 (local address 127.9.215.62:42413)
0811 18:47:55.194388 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.194391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.194504 (+ 113us) server_negotiation.cc:407] Connection header received
0811 18:47:55.194562 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.194563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.194573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.194626 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.194972 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.195540 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.196396 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.196723 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.196815 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.197794 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.197795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.197796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.197804 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.199139 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.199546 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.199547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.199547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.199613 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.199804 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.199805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.199805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.199912 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.199971 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.200102 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.200178 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 18:47:55.200299 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.194291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.9.215.1:59791)
0811 18:47:55.194440 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.194451 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.194508 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.194632 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.194633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.194818 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.194878 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.194880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.195647 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.195648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.196286 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.196289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.196837 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.196838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.196914 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.197396 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.197400 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.197710 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.199216 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.199217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.199218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.199450 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.199680 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.199681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.199682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.199749 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.199965 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.199965 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.200015 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.200084 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.200202 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:47:55.227155 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:42413
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:55.312160 12800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:55.312388 12800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:55.312414 12800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:55.313699 12800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:55.313747 12800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:55.313767 12800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:47:55.313786 12800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:47:55.315301 12800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:42413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:55.315599 12800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:55.315835 12800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:47:55.321045 12800 server_base.cc:1034] running on GCE node
W20260811 18:47:55.321059 12819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:55.321178 12821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:55.321069 12818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:55.321478 12800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:55.321729 12800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:55.322886 12800 hybrid_clock.cc:648] HybridClock initialized: now 1786474075322877 us; error 34 us; skew 500 ppm
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:55.325407 12800 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:47:55.325876 12800 webserver.cc:460] Webserver started at http://127.9.215.2:46323/ using document root <none> and password file <none>
I20260811 18:47:55.326122 12800 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:55.326184 12800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:55.326344 12800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:55.327507 12800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "756dbec1d89f4744904a1d6c72a72a1b"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "ab2f89ae254b1d1cb231ffff1e979315"
server_key_iv: "e6100f7234d66da451596202fc0c967e"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.327862 12800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "756dbec1d89f4744904a1d6c72a72a1b"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "ab2f89ae254b1d1cb231ffff1e979315"
server_key_iv: "e6100f7234d66da451596202fc0c967e"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.329228 12800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:47:55.329982 12835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.330139 12800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 18:47:55.330256 12800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "756dbec1d89f4744904a1d6c72a72a1b"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "ab2f89ae254b1d1cb231ffff1e979315"
server_key_iv: "e6100f7234d66da451596202fc0c967e"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.330330 12800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:55.346621 12800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:55.348996 12800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:55.349165 12800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:55.349783 12800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:55.349836 12800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.349872 12800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:55.349900 12800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:55.357671 12800 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:34525
I20260811 18:47:55.357725 13057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:34525 every 8 connection(s)
I20260811 18:47:55.358143 12800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:47:55.360309 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.350231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:59357 (local address 127.9.215.62:42413)
0811 18:47:55.350301 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.350304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.350554 (+ 250us) server_negotiation.cc:407] Connection header received
0811 18:47:55.351102 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.351103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.351118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.351172 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.351939 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.352479 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.353753 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.354012 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.354077 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.357104 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.357105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.357106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.357116 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.358800 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.359244 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.359246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.359246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.359314 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.359625 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.359626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.359626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.359766 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.359825 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.360116 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.360204 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:47:55.360479 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.350229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.9.215.2:59357)
0811 18:47:55.350414 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.350455 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.350992 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.351270 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.351274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.351624 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.351838 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.351844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.352571 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.352572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.353618 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.353623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.354099 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.354100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.354294 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.354864 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.354877 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.356993 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.358889 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.358895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.358897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.359167 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.359367 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.359368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.359368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.359535 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.359826 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.359835 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.359983 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.360070 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.360250 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260811 18:47:55.360996 13061 heartbeater.cc:347] Connected to a master server at 127.9.215.62:42413
I20260811 18:47:55.361104 13061 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:55.361301 13061 heartbeater.cc:511] Master 127.9.215.62:42413 requested a full tablet report, sending...
I20260811 18:47:55.361747 12383 ts_manager.cc:194] Registered new tserver with Master: 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525)
I20260811 18:47:55.362262 12383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:59357
I20260811 18:47:55.363396 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12800
I20260811 18:47:55.363516 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:47:55.363695 10076 external_mini_cluster.cc:1428] Setting key 8105a3840f613736981bd5d534bdb93f
I20260811 18:47:55.367865 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.362031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:59453 (local address 127.9.215.62:42413)
0811 18:47:55.362075 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.362077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.362195 (+ 118us) server_negotiation.cc:407] Connection header received
0811 18:47:55.362279 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.362280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.362289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.362334 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.362763 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.363267 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.364119 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.364338 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.364398 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.365417 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.365418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.365418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.365427 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.366763 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.367078 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.367080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.367080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.367134 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.367299 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.367300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.367300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.367451 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.367537 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.367681 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.367764 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:47:55.367869 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.361958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.9.215.2:59453)
0811 18:47:55.362119 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.362132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.362204 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.362382 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.362383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.362604 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.362680 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.362683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.363366 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.363367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.364037 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.364041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.364415 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.364416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.364494 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.365002 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.365008 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.365327 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.366844 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.366845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.366846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.367016 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.367188 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.367189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.367189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.367249 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.367540 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.367541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.367593 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.367650 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.367774 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:47:55.390892 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:42413
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:55.477947 13075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:55.478185 13075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:55.478214 13075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:55.479514 13075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:55.479579 13075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:55.479602 13075 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:47:55.479626 13075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:47:55.481232 13075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:42413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:55.481462 13075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:55.481678 13075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:55.486984 13083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:55.486982 13089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:55.486982 13082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:55.487715 13075 server_base.cc:1034] running on GCE node
I20260811 18:47:55.487959 13075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:55.488242 13075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:55.489498 13075 hybrid_clock.cc:648] HybridClock initialized: now 1786474075489484 us; error 30 us; skew 500 ppm
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:55.492341 13075 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:47:55.492885 13075 webserver.cc:460] Webserver started at http://127.9.215.3:39583/ using document root <none> and password file <none>
I20260811 18:47:55.493131 13075 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:55.493180 13075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:55.493310 13075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:55.494432 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "9251a0afc2bf425081aa09a40a1dac09"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "a916ceb67ef3c8a225ead627b5fa0774"
server_key_iv: "b6cf3be47ccdeb513988f06b479d9b4e"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.494769 13075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "9251a0afc2bf425081aa09a40a1dac09"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "a916ceb67ef3c8a225ead627b5fa0774"
server_key_iv: "b6cf3be47ccdeb513988f06b479d9b4e"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.496272 13075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 18:47:55.497213 13102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.497457 13075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:55.497541 13075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "9251a0afc2bf425081aa09a40a1dac09"
format_stamp: "Formatted at 2026-08-11 18:47:55 on dist-test-slave-pww0"
server_key: "a916ceb67ef3c8a225ead627b5fa0774"
server_key_iv: "b6cf3be47ccdeb513988f06b479d9b4e"
server_key_version: "encryptionkey@0"
I20260811 18:47:55.497607 13075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:55.510757 13075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:55.513126 13075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:55.513257 13075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:55.513859 13075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:55.513916 13075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.513957 13075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:55.513974 13075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:55.521440 13075 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:46057
I20260811 18:47:55.521832 13075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:47:55.521807 13336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:46057 every 8 connection(s)
I20260811 18:47:55.524159 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.514328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:40217 (local address 127.9.215.62:42413)
0811 18:47:55.514398 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.514401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.514704 (+ 303us) server_negotiation.cc:407] Connection header received
0811 18:47:55.515393 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.515395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.515408 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.515524 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.516419 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.516926 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.518459 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.518671 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.518729 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.521238 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.521239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.521240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.521249 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.522731 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.523154 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.523155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.523155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.523210 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.523550 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.523551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.523551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.523697 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.523756 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.524012 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.524086 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:47:55.524358 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.514306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.9.215.3:40217)
0811 18:47:55.514506 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.514561 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.515231 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.515620 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.515625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.515990 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.516319 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.516327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.517024 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.517025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.518361 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.518365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.518750 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.518752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.518911 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.519592 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.519601 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.521132 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.522821 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.522828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.522830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.523067 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.523294 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.523295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.523295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.523415 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.523758 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.523765 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.523903 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.523992 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.524133 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260811 18:47:55.524842 13337 heartbeater.cc:347] Connected to a master server at 127.9.215.62:42413
I20260811 18:47:55.524931 13337 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:55.525118 13337 heartbeater.cc:511] Master 127.9.215.62:42413 requested a full tablet report, sending...
I20260811 18:47:55.525518 12385 ts_manager.cc:194] Registered new tserver with Master: 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
I20260811 18:47:55.526052 12385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:40217
I20260811 18:47:55.527256 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13075
I20260811 18:47:55.527340 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:47:55.527537 10076 external_mini_cluster.cc:1428] Setting key 833ce49c54d9e2880fc0fc0d9fd02d5e
I20260811 18:47:55.528412 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:47:55.531347 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.525908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:36361 (local address 127.9.215.62:42413)
0811 18:47:55.525949 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.525951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.526019 (+ 68us) server_negotiation.cc:407] Connection header received
0811 18:47:55.526083 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.526084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.526092 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.526139 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.526506 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.527027 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.527906 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.528191 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.528256 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.529157 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.529158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.529158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.529166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.530350 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.530646 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.530648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.530648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.530704 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.530866 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.530867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.530867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.530973 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.531028 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.531177 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.531251 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:47:55.531359 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.525840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.9.215.3:36361)
0811 18:47:55.525963 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.525974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.526033 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.526154 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.526155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.526365 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.526431 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.526434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.527129 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.527130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.527820 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.527822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.528255 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.528256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.528338 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.528779 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.528783 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.529065 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.530421 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.530422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.530422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.530583 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.530755 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.530756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.530756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.530816 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.531024 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.531025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.531072 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.531143 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.531253 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:55.553188 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.546184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.9.215.62:42413)
0811 18:47:55.546280 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.546282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.546293 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:47:55.546320 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.546321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.546332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.546391 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.546941 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.547440 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.548168 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.548352 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.548399 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.550714 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.550716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.550716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.550725 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.552039 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.552392 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.552393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.552394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.552454 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.552610 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.552610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.552610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.552718 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.552790 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.553001 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.553088 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:47:55.555380 12385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:47:55.557063 12385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:55.564126 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.559787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.9.215.1:38663)
0811 18:47:55.560002 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.560008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.560019 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:47:55.560062 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.560065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.560082 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.560181 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.560386 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.561507 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.562815 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.563513 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.563580 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.563652 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.563967 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.564015 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":103,"threads_started":1}
I20260811 18:47:55.564255 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.559862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:46057 (local address 127.0.0.1:38208)
0811 18:47:55.560246 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.560257 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.560338 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.560593 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.560594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.560596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.560653 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.560654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.561897 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.561898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.563054 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.563057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.563998 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.563999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.564056 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.564060 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.564110 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.564173 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":72,"threads_started":1}
I20260811 18:47:55.564492 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.559982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.9.215.3:46057)
0811 18:47:55.560193 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.560197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.560310 (+ 113us) server_negotiation.cc:407] Connection header received
0811 18:47:55.560433 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.560435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.560451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.560500 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.560736 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.561815 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.563154 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.563921 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.563987 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.564043 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.564140 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.564179 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":79,"threads_started":1}
I20260811 18:47:55.564692 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.559663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38663 (local address 127.0.0.1:46154)
0811 18:47:55.559792 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.559835 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.559901 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.560187 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.560191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.560194 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.560285 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.560287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.561594 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.561595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.562733 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.562736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.563722 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.563723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.563827 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.563845 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.564419 (+ 574us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.564474 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
I20260811 18:47:55.565097 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.560016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:34525 (local address 127.0.0.1:37886)
0811 18:47:55.560314 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.560323 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.560364 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.561836 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.561837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.561839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.561877 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.561879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.563165 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.563166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.563927 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.563929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.564875 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.564876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.564936 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.564940 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.564984 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.565024 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":69,"threads_started":1}
I20260811 18:47:55.565202 13199 tablet_service.cc:1467] Processing CreateTablet for tablet e9b9dc5289cd43d5a8557efaf854df1c (DEFAULT_TABLE table=test-table [id=5ac64c4fa7d6443e96d2d43c9b967223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:55.565342 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.560144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.9.215.2:34525)
0811 18:47:55.561672 (+ 1528us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.561677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.561686 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:47:55.561724 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.561726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.561742 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.561815 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.561961 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.563079 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.564015 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.564717 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.564784 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.564844 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.565188 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.565246 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1487,"thread_start_us":84,"threads_started":1}
I20260811 18:47:55.565415 12656 tablet_service.cc:1467] Processing CreateTablet for tablet e9b9dc5289cd43d5a8557efaf854df1c (DEFAULT_TABLE table=test-table [id=5ac64c4fa7d6443e96d2d43c9b967223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:55.565610 13199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9b9dc5289cd43d5a8557efaf854df1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.565754 12656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9b9dc5289cd43d5a8557efaf854df1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.565914 12925 tablet_service.cc:1467] Processing CreateTablet for tablet e9b9dc5289cd43d5a8557efaf854df1c (DEFAULT_TABLE table=test-table [id=5ac64c4fa7d6443e96d2d43c9b967223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:55.566198 12925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9b9dc5289cd43d5a8557efaf854df1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.569124 13379 tablet_bootstrap.cc:492] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Bootstrap starting.
I20260811 18:47:55.569521 13381 tablet_bootstrap.cc:492] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Bootstrap starting.
I20260811 18:47:55.569624 13380 tablet_bootstrap.cc:492] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Bootstrap starting.
I20260811 18:47:55.570094 13381 tablet_bootstrap.cc:654] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.570185 13379 tablet_bootstrap.cc:654] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.570222 13380 tablet_bootstrap.cc:654] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.570573 13379 log.cc:824] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:55.570590 13380 log.cc:824] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:55.571075 13381 log.cc:824] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:55.571537 13379 tablet_bootstrap.cc:492] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: No bootstrap required, opened a new log
I20260811 18:47:55.571614 13379 ts_tablet_manager.cc:1397] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:47:55.571797 13380 tablet_bootstrap.cc:492] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: No bootstrap required, opened a new log
I20260811 18:47:55.571874 13380 ts_tablet_manager.cc:1397] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:47:55.572120 13381 tablet_bootstrap.cc:492] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: No bootstrap required, opened a new log
I20260811 18:47:55.572230 13381 ts_tablet_manager.cc:1397] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:47:55.573074 13379 raft_consensus.cc:348] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.573226 13379 raft_consensus.cc:374] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.573251 13379 raft_consensus.cc:729] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9251a0afc2bf425081aa09a40a1dac09, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.573354 13379 consensus_queue.cc:260] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.573647 13337 heartbeater.cc:503] Master 127.9.215.62:42413 was elected leader, sending a full tablet report...
I20260811 18:47:55.573880 13379 ts_tablet_manager.cc:1428] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:47:55.573875 13380 raft_consensus.cc:348] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.574193 13380 raft_consensus.cc:374] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.574244 13380 raft_consensus.cc:729] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756dbec1d89f4744904a1d6c72a72a1b, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.574358 13380 consensus_queue.cc:260] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.574637 13380 ts_tablet_manager.cc:1428] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:47:55.574770 13061 heartbeater.cc:503] Master 127.9.215.62:42413 was elected leader, sending a full tablet report...
I20260811 18:47:55.574769 13381 raft_consensus.cc:348] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.574937 13381 raft_consensus.cc:374] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.574976 13381 raft_consensus.cc:729] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a97a06fa504ca1a9a6f4376617062e, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.575111 13381 consensus_queue.cc:260] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.575399 13381 ts_tablet_manager.cc:1428] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:47:55.575563 12786 heartbeater.cc:503] Master 127.9.215.62:42413 was elected leader, sending a full tablet report...
W20260811 18:47:55.608880 13063 tablet.cc:2369] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:47:55.686062 13388 raft_consensus.cc:482] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:55.686214 13388 raft_consensus.cc:504] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.686594 13388 leader_election.cc:288] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1:38663), 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
W20260811 18:47:55.690049 12788 tablet.cc:2369] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:47:55.690696 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.687181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:55061 (local address 127.9.215.3:46057)
0811 18:47:55.687238 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.687240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.687413 (+ 173us) server_negotiation.cc:407] Connection header received
0811 18:47:55.687486 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.687487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.687501 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.687566 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.687703 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.688477 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.689787 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.690317 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.690383 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.690396 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.690578 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.690627 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:47:55.690732 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.686911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:45607 (local address 127.9.215.1:38663)
0811 18:47:55.686995 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.686997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.687064 (+ 67us) server_negotiation.cc:407] Connection header received
0811 18:47:55.687167 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.687168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.687185 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.687253 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.687483 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.688263 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.689760 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.690188 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.690257 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.690286 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.690601 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.690677 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:47:55.690706 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.686922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38663 (local address 127.9.215.2:45607)
0811 18:47:55.686976 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.686988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.687088 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.687263 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.687264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.687266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.687354 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.687356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.688365 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.688367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.689642 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.689645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.690324 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.690325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.690517 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.690520 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.690574 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.690623 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 18:47:55.690732 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.687081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:46057 (local address 127.9.215.2:55061)
0811 18:47:55.687317 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.687326 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.687385 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.687568 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.687568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.687570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.687638 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.687639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.688561 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.688562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.689647 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.689648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.690399 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.690400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.690483 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.690487 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.690556 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.690607 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":186,"threads_started":1}
I20260811 18:47:55.690979 13389 raft_consensus.cc:482] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:55.691092 13389 raft_consensus.cc:504] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.691220 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b9dc5289cd43d5a8557efaf854df1c" candidate_uuid: "756dbec1d89f4744904a1d6c72a72a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09" is_pre_election: true
I20260811 18:47:55.691248 12691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b9dc5289cd43d5a8557efaf854df1c" candidate_uuid: "756dbec1d89f4744904a1d6c72a72a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a97a06fa504ca1a9a6f4376617062e" is_pre_election: true
I20260811 18:47:55.691354 12691 raft_consensus.cc:2454] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 756dbec1d89f4744904a1d6c72a72a1b in term 0.
I20260811 18:47:55.691367 13227 raft_consensus.cc:2454] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 756dbec1d89f4744904a1d6c72a72a1b in term 0.
I20260811 18:47:55.691481 13389 leader_election.cc:288] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057), 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525)
I20260811 18:47:55.691661 12841 leader_election.cc:302] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756dbec1d89f4744904a1d6c72a72a1b, a1a97a06fa504ca1a9a6f4376617062e; no voters:
I20260811 18:47:55.691877 13388 raft_consensus.cc:2781] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:55.691933 13388 raft_consensus.cc:482] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:55.691956 13388 raft_consensus.cc:3037] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.693475 13388 raft_consensus.cc:504] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.693601 13388 leader_election.cc:288] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [CANDIDATE]: Term 1 election: Requested vote from peers a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1:38663), 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
I20260811 18:47:55.693810 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b9dc5289cd43d5a8557efaf854df1c" candidate_uuid: "756dbec1d89f4744904a1d6c72a72a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09"
I20260811 18:47:55.693809 12691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b9dc5289cd43d5a8557efaf854df1c" candidate_uuid: "756dbec1d89f4744904a1d6c72a72a1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a97a06fa504ca1a9a6f4376617062e"
I20260811 18:47:55.693889 12691 raft_consensus.cc:3037] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.693892 13227 raft_consensus.cc:3037] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.694756 13227 raft_consensus.cc:2454] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 756dbec1d89f4744904a1d6c72a72a1b in term 1.
I20260811 18:47:55.694756 12691 raft_consensus.cc:2454] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 756dbec1d89f4744904a1d6c72a72a1b in term 1.
I20260811 18:47:55.694891 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.691623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:46057 (local address 127.9.215.1:44635)
0811 18:47:55.691675 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.691686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.691722 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.691987 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.691988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.691990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.692050 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.692052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.692671 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.692672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.693875 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.693878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.694647 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.694648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.694728 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.694732 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.694777 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.694819 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 18:47:55.694943 12841 leader_election.cc:302] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756dbec1d89f4744904a1d6c72a72a1b, a1a97a06fa504ca1a9a6f4376617062e; no voters:
I20260811 18:47:55.695043 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.691805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:60995 (local address 127.9.215.2:34525)
0811 18:47:55.691843 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.691845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.691941 (+ 96us) server_negotiation.cc:407] Connection header received
0811 18:47:55.691987 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.691988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.691997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.692043 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.692225 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.692899 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.694167 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.694710 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.694750 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.694759 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.694951 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.694990 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:47:55.695066 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.691804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:44635 (local address 127.9.215.3:46057)
0811 18:47:55.691870 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.691872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.691878 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:47:55.691897 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.691897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.691904 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.691947 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:55.692119 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.692595 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.693987 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.694548 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.694606 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.694617 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.694979 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.695016 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:47:55.695061 13388 raft_consensus.cc:2781] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:55.695245 13388 raft_consensus.cc:686] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 1 LEADER]: Becoming Leader. State: Replica: 756dbec1d89f4744904a1d6c72a72a1b, State: Running, Role: LEADER
I20260811 18:47:55.695271 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.691741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:34525 (local address 127.9.215.1:60995)
0811 18:47:55.691870 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.691879 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.691940 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.692080 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.692080 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.692082 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:55.692149 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.692150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.692978 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.692979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.694081 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.694083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.694811 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.694812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.694875 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.694878 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.695151 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.695194 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260811 18:47:55.695346 13388 consensus_queue.cc:237] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } }
I20260811 18:47:55.695693 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b9dc5289cd43d5a8557efaf854df1c" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756dbec1d89f4744904a1d6c72a72a1b" is_pre_election: true
I20260811 18:47:55.696028 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9b9dc5289cd43d5a8557efaf854df1c" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09" is_pre_election: true
I20260811 18:47:55.696142 13227 raft_consensus.cc:2379] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1a97a06fa504ca1a9a6f4376617062e in current term 1: Already voted for candidate 756dbec1d89f4744904a1d6c72a72a1b in this term.
I20260811 18:47:55.696271 12383 catalog_manager.cc:5495] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2). New cstate: current_term: 1 leader_uuid: "756dbec1d89f4744904a1d6c72a72a1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } health_report { overall_health: HEALTHY } } }
I20260811 18:47:55.697266 12552 leader_election.cc:302] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1a97a06fa504ca1a9a6f4376617062e; no voters: 756dbec1d89f4744904a1d6c72a72a1b, 9251a0afc2bf425081aa09a40a1dac09
I20260811 18:47:55.697420 13389 raft_consensus.cc:2726] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:47:55.712018 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.709072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37900 (local address 127.9.215.2:34525)
0811 18:47:55.709137 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.709139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.709147 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:47:55.709213 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.709214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.709233 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.709295 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:55.709476 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.709966 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.710932 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.711145 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.711222 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.711558 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:55.711723 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:55.711793 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.711895 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.711920 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:47:55.714392 12691 raft_consensus.cc:1261] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Refusing update from remote peer 756dbec1d89f4744904a1d6c72a72a1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:55.714370 13227 raft_consensus.cc:1261] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Refusing update from remote peer 756dbec1d89f4744904a1d6c72a72a1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:55.714740 13388 consensus_queue.cc:1035] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:55.714903 13398 consensus_queue.cc:1035] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:55.719551 13408 mvcc.cc:205] Tried to move back new op lower bound from 7317397814122004480 to 7317397814048821248. Current Snapshot: MvccSnapshot[applied={T|T < 7317397814122004480}]
I20260811 18:47:55.720117 13403 mvcc.cc:205] Tried to move back new op lower bound from 7317397814122004480 to 7317397814048821248. Current Snapshot: MvccSnapshot[applied={T|T < 7317397814122004480}]
I20260811 18:47:55.724150 12383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:47:55.724370 12383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:47:55.725395 12383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: Sending DeleteTablet for 3 replicas of tablet e9b9dc5289cd43d5a8557efaf854df1c
I20260811 18:47:55.725941 12925 tablet_service.cc:1514] Processing DeleteTablet for tablet e9b9dc5289cd43d5a8557efaf854df1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:55 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:37886
I20260811 18:47:55.725924 12656 tablet_service.cc:1514] Processing DeleteTablet for tablet e9b9dc5289cd43d5a8557efaf854df1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:55 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:46154
I20260811 18:47:55.726039 13199 tablet_service.cc:1514] Processing DeleteTablet for tablet e9b9dc5289cd43d5a8557efaf854df1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:55 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:38208
I20260811 18:47:55.726745 13420 tablet_replica.cc:330] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: stopping tablet replica
I20260811 18:47:55.726886 13420 raft_consensus.cc:2229] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:47:55.726841 12383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:47:55.727021 12383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:55.727052 13420 raft_consensus.cc:2258] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:55.727244 13418 tablet_replica.cc:330] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: stopping tablet replica
I20260811 18:47:55.727407 13418 raft_consensus.cc:2229] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:47:55.727604 13420 ts_tablet_manager.cc:1905] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:55.727636 13418 raft_consensus.cc:2258] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:55.728444 13416 tablet_replica.cc:330] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: stopping tablet replica
I20260811 18:47:55.728603 12656 tablet_service.cc:1467] Processing CreateTablet for tablet 8faed0cf986b401fb75748731b1d26be (DEFAULT_TABLE table=test-table [id=ecc1cbc108e44709b5cfa6c4957f25ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:55.728645 13416 raft_consensus.cc:2229] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:47:55.728758 12656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8faed0cf986b401fb75748731b1d26be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.728921 13416 raft_consensus.cc:2258] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:55.729743 13420 ts_tablet_manager.cc:1918] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:47:55.729827 13420 log.cc:1196] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/e9b9dc5289cd43d5a8557efaf854df1c
I20260811 18:47:55.730008 13416 ts_tablet_manager.cc:1905] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:55.730096 13420 ts_tablet_manager.cc:1939] T e9b9dc5289cd43d5a8557efaf854df1c P 9251a0afc2bf425081aa09a40a1dac09: Deleting consensus metadata
I20260811 18:47:55.730661 12925 tablet_service.cc:1467] Processing CreateTablet for tablet 8faed0cf986b401fb75748731b1d26be (DEFAULT_TABLE table=test-table [id=ecc1cbc108e44709b5cfa6c4957f25ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:55.730795 12925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8faed0cf986b401fb75748731b1d26be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.731570 13416 ts_tablet_manager.cc:1918] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:47:55.731645 13416 log.cc:1196] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/e9b9dc5289cd43d5a8557efaf854df1c
I20260811 18:47:55.731896 13416 ts_tablet_manager.cc:1939] T e9b9dc5289cd43d5a8557efaf854df1c P a1a97a06fa504ca1a9a6f4376617062e: Deleting consensus metadata
I20260811 18:47:55.732540 13380 tablet_bootstrap.cc:492] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Bootstrap starting.
I20260811 18:47:55.732571 13381 tablet_bootstrap.cc:492] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Bootstrap starting.
I20260811 18:47:55.733067 13380 tablet_bootstrap.cc:654] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.733408 13381 tablet_bootstrap.cc:654] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.734308 13380 tablet_bootstrap.cc:492] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: No bootstrap required, opened a new log
I20260811 18:47:55.734355 13380 ts_tablet_manager.cc:1397] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:47:55.734509 12354 catalog_manager.cc:4841] TS 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057): tablet e9b9dc5289cd43d5a8557efaf854df1c (table test-table [id=5ac64c4fa7d6443e96d2d43c9b967223]) successfully deleted
I20260811 18:47:55.734674 12354 catalog_manager.cc:4841] TS a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1:38663): tablet e9b9dc5289cd43d5a8557efaf854df1c (table test-table [id=5ac64c4fa7d6443e96d2d43c9b967223]) successfully deleted
I20260811 18:47:55.734661 13199 tablet_service.cc:1467] Processing CreateTablet for tablet 8faed0cf986b401fb75748731b1d26be (DEFAULT_TABLE table=test-table [id=ecc1cbc108e44709b5cfa6c4957f25ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:55.734704 13380 raft_consensus.cc:348] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.734784 13380 raft_consensus.cc:374] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.734786 13199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8faed0cf986b401fb75748731b1d26be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.734803 13380 raft_consensus.cc:729] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756dbec1d89f4744904a1d6c72a72a1b, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.734848 13380 consensus_queue.cc:260] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.734934 13380 ts_tablet_manager.cc:1428] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:55.735304 13418 ts_tablet_manager.cc:1905] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:55.736369 13379 tablet_bootstrap.cc:492] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Bootstrap starting.
I20260811 18:47:55.736770 13381 tablet_bootstrap.cc:492] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: No bootstrap required, opened a new log
I20260811 18:47:55.736824 13381 ts_tablet_manager.cc:1397] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:47:55.737183 13381 raft_consensus.cc:348] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.737277 13381 raft_consensus.cc:374] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.737320 13381 raft_consensus.cc:729] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a97a06fa504ca1a9a6f4376617062e, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.737375 13381 consensus_queue.cc:260] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.737471 13381 ts_tablet_manager.cc:1428] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:47:55.737488 13379 tablet_bootstrap.cc:654] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.737494 13418 ts_tablet_manager.cc:1918] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:47:55.737560 13418 log.cc:1196] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/e9b9dc5289cd43d5a8557efaf854df1c
I20260811 18:47:55.737824 13418 ts_tablet_manager.cc:1939] T e9b9dc5289cd43d5a8557efaf854df1c P 756dbec1d89f4744904a1d6c72a72a1b: Deleting consensus metadata
I20260811 18:47:55.738960 13379 tablet_bootstrap.cc:492] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: No bootstrap required, opened a new log
I20260811 18:47:55.739028 13379 ts_tablet_manager.cc:1397] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:47:55.739614 13379 raft_consensus.cc:348] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.739701 13379 raft_consensus.cc:374] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.739797 13379 raft_consensus.cc:729] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9251a0afc2bf425081aa09a40a1dac09, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.739881 13379 consensus_queue.cc:260] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.739984 13379 ts_tablet_manager.cc:1428] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:47:55.740761 12354 catalog_manager.cc:4841] TS 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525): tablet e9b9dc5289cd43d5a8557efaf854df1c (table test-table [id=5ac64c4fa7d6443e96d2d43c9b967223]) successfully deleted
W20260811 18:47:55.772719 13338 tablet.cc:2369] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:47:55.799559 13389 raft_consensus.cc:482] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:55.799690 13389 raft_consensus.cc:504] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.799889 13389 leader_election.cc:288] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525), 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
I20260811 18:47:55.800209 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8faed0cf986b401fb75748731b1d26be" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756dbec1d89f4744904a1d6c72a72a1b" is_pre_election: true
I20260811 18:47:55.800266 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8faed0cf986b401fb75748731b1d26be" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09" is_pre_election: true
I20260811 18:47:55.800352 13227 raft_consensus.cc:2454] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 0.
I20260811 18:47:55.800348 12962 raft_consensus.cc:2454] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 0.
I20260811 18:47:55.800563 12552 leader_election.cc:302] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756dbec1d89f4744904a1d6c72a72a1b, a1a97a06fa504ca1a9a6f4376617062e; no voters:
I20260811 18:47:55.800697 13389 raft_consensus.cc:2781] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:55.800740 13389 raft_consensus.cc:482] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:55.800757 13389 raft_consensus.cc:3037] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.802095 13389 raft_consensus.cc:504] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.802232 13389 leader_election.cc:288] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 election: Requested vote from peers 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525), 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
I20260811 18:47:55.802466 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8faed0cf986b401fb75748731b1d26be" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756dbec1d89f4744904a1d6c72a72a1b"
I20260811 18:47:55.802481 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8faed0cf986b401fb75748731b1d26be" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09"
I20260811 18:47:55.802556 12962 raft_consensus.cc:3037] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.802577 13227 raft_consensus.cc:3037] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.803192 12962 raft_consensus.cc:2454] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 1.
I20260811 18:47:55.803480 12552 leader_election.cc:302] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756dbec1d89f4744904a1d6c72a72a1b, a1a97a06fa504ca1a9a6f4376617062e; no voters:
I20260811 18:47:55.803594 13389 raft_consensus.cc:2781] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:55.803735 13389 raft_consensus.cc:686] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 1 LEADER]: Becoming Leader. State: Replica: a1a97a06fa504ca1a9a6f4376617062e, State: Running, Role: LEADER
I20260811 18:47:55.803823 13389 consensus_queue.cc:237] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.804103 13227 raft_consensus.cc:2454] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 1.
I20260811 18:47:55.804654 12385 catalog_manager.cc:5495] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e reported cstate change: term changed from 0 to 1, leader changed from <none> to a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1). New cstate: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:47:55 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474074, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:55.857551 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.848511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.9.215.62:42413)
0811 18:47:55.848588 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.848590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.848846 (+ 256us) server_negotiation.cc:407] Connection header received
0811 18:47:55.848934 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.848934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.848945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.848986 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.849808 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.850329 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.851372 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.851625 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.851690 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.854664 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:55.854665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:55.854665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:55.854673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:55.856091 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.856622 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.856624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.856624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.856694 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:55.856947 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:55.856948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:55.856948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:55.857078 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:55.857144 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.857360 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.857479 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":47}
I20260811 18:47:55.857673 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.848461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:42413 (local address 127.0.0.1:55502)
0811 18:47:55.848751 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.848766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.848857 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.849022 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.849023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.849617 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:55.849688 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.849691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.850419 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.850420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.851257 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.851260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.851706 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.851707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.851777 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.852427 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:55.852433 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:55.854489 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:55.856187 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.856203 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.856206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.856522 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.856770 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:55.856771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:55.856772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:55.856852 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:55.857156 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:55.857168 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:55.857232 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.857325 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.857569 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":96,"thread_start_us":144,"threads_started":1}
I20260811 18:47:55.860716 12385 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:55502:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:47:55.861049 12385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:55.862586 12656 tablet_service.cc:1467] Processing CreateTablet for tablet 265cc18f57bc499bbeffbcbbfb841bc0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe4780042e0746e689a40ed09bee5a39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:47:55.862638 13199 tablet_service.cc:1467] Processing CreateTablet for tablet 265cc18f57bc499bbeffbcbbfb841bc0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe4780042e0746e689a40ed09bee5a39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:47:55.862795 12656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265cc18f57bc499bbeffbcbbfb841bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.862805 13199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265cc18f57bc499bbeffbcbbfb841bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.864500 13379 tablet_bootstrap.cc:492] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: Bootstrap starting.
I20260811 18:47:55.864759 13381 tablet_bootstrap.cc:492] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: Bootstrap starting.
I20260811 18:47:55.864835 12925 tablet_service.cc:1467] Processing CreateTablet for tablet 265cc18f57bc499bbeffbcbbfb841bc0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe4780042e0746e689a40ed09bee5a39]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:47:55.864987 12925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265cc18f57bc499bbeffbcbbfb841bc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:55.865836 13379 tablet_bootstrap.cc:654] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.866135 13381 tablet_bootstrap.cc:654] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.866711 13380 tablet_bootstrap.cc:492] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: Bootstrap starting.
I20260811 18:47:55.867600 13379 tablet_bootstrap.cc:492] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: No bootstrap required, opened a new log
I20260811 18:47:55.867651 13379 ts_tablet_manager.cc:1397] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:47:55.868121 13379 raft_consensus.cc:348] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.868204 13379 raft_consensus.cc:374] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.868225 13379 raft_consensus.cc:729] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9251a0afc2bf425081aa09a40a1dac09, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.868270 13379 consensus_queue.cc:260] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.868335 13380 tablet_bootstrap.cc:654] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:55.868360 13379 ts_tablet_manager.cc:1428] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:47:55.868561 13386 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.868693 13386 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.869984 13380 tablet_bootstrap.cc:492] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: No bootstrap required, opened a new log
I20260811 18:47:55.870045 13380 ts_tablet_manager.cc:1397] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:47:55.870299 13380 raft_consensus.cc:348] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.870388 13380 raft_consensus.cc:374] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.870416 13380 raft_consensus.cc:729] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756dbec1d89f4744904a1d6c72a72a1b, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.870463 13380 consensus_queue.cc:260] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.870553 13380 ts_tablet_manager.cc:1428] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:55.870606 13388 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.870694 13388 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.871111 13381 tablet_bootstrap.cc:492] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: No bootstrap required, opened a new log
I20260811 18:47:55.871186 13381 ts_tablet_manager.cc:1397] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:47:55.871610 13381 raft_consensus.cc:348] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.871699 13381 raft_consensus.cc:374] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:55.871717 13381 raft_consensus.cc:729] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a97a06fa504ca1a9a6f4376617062e, State: Initialized, Role: FOLLOWER
I20260811 18:47:55.871757 13381 consensus_queue.cc:260] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.871832 13381 ts_tablet_manager.cc:1428] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:47:55.871932 13389 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.872027 13389 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.924269 13389 raft_consensus.cc:482] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:55.924410 13389 raft_consensus.cc:504] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.924700 13389 leader_election.cc:288] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525), 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
I20260811 18:47:55.924932 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265cc18f57bc499bbeffbcbbfb841bc0" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756dbec1d89f4744904a1d6c72a72a1b" is_pre_election: true
I20260811 18:47:55.924939 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265cc18f57bc499bbeffbcbbfb841bc0" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09" is_pre_election: true
I20260811 18:47:55.925067 13227 raft_consensus.cc:2454] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 0.
I20260811 18:47:55.925063 12962 raft_consensus.cc:2454] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 0.
I20260811 18:47:55.925253 12552 leader_election.cc:302] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9251a0afc2bf425081aa09a40a1dac09, a1a97a06fa504ca1a9a6f4376617062e; no voters:
I20260811 18:47:55.925364 13389 raft_consensus.cc:2781] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:55.925436 13389 raft_consensus.cc:482] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:55.925454 13389 raft_consensus.cc:3037] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.926798 13389 raft_consensus.cc:504] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.926939 13389 leader_election.cc:288] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 election: Requested vote from peers 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525), 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057)
I20260811 18:47:55.927148 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265cc18f57bc499bbeffbcbbfb841bc0" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "756dbec1d89f4744904a1d6c72a72a1b"
I20260811 18:47:55.927165 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265cc18f57bc499bbeffbcbbfb841bc0" candidate_uuid: "a1a97a06fa504ca1a9a6f4376617062e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9251a0afc2bf425081aa09a40a1dac09"
I20260811 18:47:55.927229 12962 raft_consensus.cc:3037] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.927254 13227 raft_consensus.cc:3037] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:55.928037 12962 raft_consensus.cc:2454] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 1.
I20260811 18:47:55.928246 12552 leader_election.cc:302] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 756dbec1d89f4744904a1d6c72a72a1b, a1a97a06fa504ca1a9a6f4376617062e; no voters:
I20260811 18:47:55.928238 13227 raft_consensus.cc:2454] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1a97a06fa504ca1a9a6f4376617062e in term 1.
I20260811 18:47:55.928385 13389 raft_consensus.cc:2781] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:55.928452 13389 raft_consensus.cc:686] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [term 1 LEADER]: Becoming Leader. State: Replica: a1a97a06fa504ca1a9a6f4376617062e, State: Running, Role: LEADER
I20260811 18:47:55.928511 13389 consensus_queue.cc:237] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } }
I20260811 18:47:55.928942 13427 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: TxnStatusTablet state changed. Reason: New leader a1a97a06fa504ca1a9a6f4376617062e. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.929136 13427 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:47:55.929217 12385 catalog_manager.cc:5495] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e reported cstate change: term changed from 0 to 1, leader changed from <none> to a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1). New cstate: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } health_report { overall_health: UNKNOWN } } }
I20260811 18:47:55.929510 13462 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:47:55.929668 13462 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:47:55.950133 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.946891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.9.215.1:38663)
0811 18:47:55.946934 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.946936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.946982 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:47:55.947093 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.947093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.947112 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.947168 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:55.947304 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.948058 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.949167 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.949458 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.949536 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.949751 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:55.949884 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:55.949953 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.950024 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.950049 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:47:55.950172 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.946826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38663 (local address 127.0.0.1:46168)
0811 18:47:55.946919 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.946933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.947006 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.947185 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.947186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.947188 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:55.947246 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.947248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.948150 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.948152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.949065 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.949069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.949543 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.949544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.949626 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.949648 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:55.949960 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:55.949970 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.950022 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.950085 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 18:47:55.951982 13227 raft_consensus.cc:1261] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Refusing update from remote peer a1a97a06fa504ca1a9a6f4376617062e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:55.952051 12962 raft_consensus.cc:1261] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Refusing update from remote peer a1a97a06fa504ca1a9a6f4376617062e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:55.952293 13427 consensus_queue.cc:1035] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [LEADER]: Connected to new peer: Peer: permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:55.952450 13389 consensus_queue.cc:1035] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:55.953284 13388 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: TxnStatusTablet state changed. Reason: New leader a1a97a06fa504ca1a9a6f4376617062e. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.953305 13386 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: TxnStatusTablet state changed. Reason: New leader a1a97a06fa504ca1a9a6f4376617062e. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.953383 13386 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.953382 13388 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.954267 13389 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.954368 13389 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:47:55.954856 13427 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.954950 13427 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P a1a97a06fa504ca1a9a6f4376617062e: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:47:55.955384 13386 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.955515 13386 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P 9251a0afc2bf425081aa09a40a1dac09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.956537 13388 tablet_replica.cc:439] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1a97a06fa504ca1a9a6f4376617062e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a97a06fa504ca1a9a6f4376617062e" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38663 } } peers { permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 } } peers { permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 } } }
I20260811 18:47:55.956645 13388 tablet_replica.cc:442] T 265cc18f57bc499bbeffbcbbfb841bc0 P 756dbec1d89f4744904a1d6c72a72a1b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:55.959718 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.957034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.9.215.1:38663)
0811 18:47:55.957072 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.957074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.957080 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:47:55.957102 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.957103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.957113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.957163 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:55.957299 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.958058 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.958933 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.959239 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.959298 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.959441 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:55.959502 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:55.959553 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.959617 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.959639 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:47:55.967221 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.963771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:56869 (local address 127.9.215.1:38663)
0811 18:47:55.963920 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.963922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.964104 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:47:55.964130 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.964131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.964141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.964228 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:55.964700 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.965326 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.966474 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.966657 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.966711 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.966953 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:55.966984 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:55.967026 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:55.967083 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 18:47:55.967123 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":50}
I20260811 18:47:55.967370 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.963581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38663 (local address 127.9.215.1:56869)
0811 18:47:55.963984 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:55.963996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:55.964066 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:55.964248 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:55.964249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:55.964403 (+ 154us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:55.964553 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.964555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.965424 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.965425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.966387 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:55.966390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:55.966717 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:55.966718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:55.966843 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:55.966871 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:55.967032 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:55.967034 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:47:55.967082 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:47:55.967146 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":5,"thread_start_us":229,"threads_started":1}
I20260811 18:47:55.970083 13227 raft_consensus.cc:1261] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Refusing update from remote peer a1a97a06fa504ca1a9a6f4376617062e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:55.970153 12962 raft_consensus.cc:1261] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Refusing update from remote peer a1a97a06fa504ca1a9a6f4376617062e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:55.970333 13427 consensus_queue.cc:1035] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9251a0afc2bf425081aa09a40a1dac09" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:55.970577 13427 consensus_queue.cc:1035] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [LEADER]: Connected to new peer: Peer: permanent_uuid: "756dbec1d89f4744904a1d6c72a72a1b" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:55.972314 13402 mvcc.cc:205] Tried to move back new op lower bound from 7317397815171485696 to 7317397814493134848. Current Snapshot: MvccSnapshot[applied={T|T < 7317397815171485696}]
I20260811 18:47:55.992216 12383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:47:55.992347 12383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:47:55.993095 12383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9f396f37b6d4ec2a284ce726612a69a: Sending DeleteTablet for 3 replicas of tablet 8faed0cf986b401fb75748731b1d26be
I20260811 18:47:55.993445 12925 tablet_service.cc:1514] Processing DeleteTablet for tablet 8faed0cf986b401fb75748731b1d26be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:55 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:37886
I20260811 18:47:55.993569 13418 tablet_replica.cc:330] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: stopping tablet replica
I20260811 18:47:55.993641 13418 raft_consensus.cc:2229] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:47:55.993685 13418 raft_consensus.cc:2258] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:55.993686 13199 tablet_service.cc:1514] Processing DeleteTablet for tablet 8faed0cf986b401fb75748731b1d26be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:55 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:38208
I20260811 18:47:55.993808 13420 tablet_replica.cc:330] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: stopping tablet replica
I20260811 18:47:55.993863 13420 raft_consensus.cc:2229] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:47:55.993911 13420 raft_consensus.cc:2258] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:55.994125 13418 ts_tablet_manager.cc:1905] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:55.994782 12656 tablet_service.cc:1514] Processing DeleteTablet for tablet 8faed0cf986b401fb75748731b1d26be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:55 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:46154
I20260811 18:47:55.994895 13416 tablet_replica.cc:330] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: stopping tablet replica
I20260811 18:47:55.994951 13416 raft_consensus.cc:2229] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:47:55.995031 13416 raft_consensus.cc:2258] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:55.995486 13416 ts_tablet_manager.cc:1905] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:55.995796 13420 ts_tablet_manager.cc:1905] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:55.996279 13418 ts_tablet_manager.cc:1918] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:47:55.996330 13418 log.cc:1196] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/8faed0cf986b401fb75748731b1d26be
I20260811 18:47:55.996630 13418 ts_tablet_manager.cc:1939] T 8faed0cf986b401fb75748731b1d26be P 756dbec1d89f4744904a1d6c72a72a1b: Deleting consensus metadata
I20260811 18:47:55.997025 12354 catalog_manager.cc:4841] TS 756dbec1d89f4744904a1d6c72a72a1b (127.9.215.2:34525): tablet 8faed0cf986b401fb75748731b1d26be (table test-table [id=ecc1cbc108e44709b5cfa6c4957f25ce]) successfully deleted
I20260811 18:47:55.997396 13416 ts_tablet_manager.cc:1918] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:47:55.997449 13416 log.cc:1196] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/8faed0cf986b401fb75748731b1d26be
I20260811 18:47:55.997530 13420 ts_tablet_manager.cc:1918] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:47:55.997575 13420 log.cc:1196] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/8faed0cf986b401fb75748731b1d26be
I20260811 18:47:55.997653 13416 ts_tablet_manager.cc:1939] T 8faed0cf986b401fb75748731b1d26be P a1a97a06fa504ca1a9a6f4376617062e: Deleting consensus metadata
I20260811 18:47:55.997713 13420 ts_tablet_manager.cc:1939] T 8faed0cf986b401fb75748731b1d26be P 9251a0afc2bf425081aa09a40a1dac09: Deleting consensus metadata
I20260811 18:47:55.998191 12354 catalog_manager.cc:4841] TS 9251a0afc2bf425081aa09a40a1dac09 (127.9.215.3:46057): tablet 8faed0cf986b401fb75748731b1d26be (table test-table [id=ecc1cbc108e44709b5cfa6c4957f25ce]) successfully deleted
I20260811 18:47:55.998606 12354 catalog_manager.cc:4841] TS a1a97a06fa504ca1a9a6f4376617062e (127.9.215.1:38663): tablet 8faed0cf986b401fb75748731b1d26be (table test-table [id=ecc1cbc108e44709b5cfa6c4957f25ce]) successfully deleted
Aug 11 18:47:56 dist-test-slave-pww0 krb5kdc[12296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474075, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:47:56.008723 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:55.995121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.9.215.1:38663)
0811 18:47:55.995214 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:55.995216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:55.998613 (+ 3397us) server_negotiation.cc:407] Connection header received
0811 18:47:55.998641 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:55.998642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:55.998653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:55.998700 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:55.999514 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:56.000072 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:56.001312 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:56.001575 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:56.001643 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:56.005646 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:56.005663 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:56.005664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:56.005673 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:56.007431 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:56.007862 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:56.007867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:56.007868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:56.007930 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:56.008123 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:56.008124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:56.008125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:56.008305 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:56.008471 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:56.008518 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 18:47:56.008612 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 18:47:56.009068 12772 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46190
I20260811 18:47:56.017666 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:56.010276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.9.215.62:42413)
0811 18:47:56.010345 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:56.010347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:56.010418 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:47:56.010489 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:56.010490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:56.010500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:56.010562 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:56.011272 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:56.011880 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:56.012912 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:56.013114 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:56.013178 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:56.014871 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:56.014873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:56.014873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:56.014881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:56.016376 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:56.016814 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:56.016815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:56.016816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:56.016883 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:56.017162 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:56.017164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:56.017164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:56.017299 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:56.017387 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:56.017517 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:47:56.017594 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 18:47:56.018024 12383 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55508
I20260811 18:47:56.018748 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12520
I20260811 18:47:56.035902 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12800
I20260811 18:47:56.059770 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13075
I20260811 18:47:56.081393 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12327
2026-08-11T18:47:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2632 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:47:56 dist-test-slave-pww0 krb5kdc[13497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:47:56 dist-test-slave-pww0 krb5kdc[13497](info): set up 2 sockets
Aug 11 18:47:56 dist-test-slave-pww0 krb5kdc[13497](info): commencing operation
krb5kdc: starting...
W20260811 18:47:57.323612 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:47:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:47:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:47:57.454633 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43649
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:43649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:57.539430 13514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:57.539711 13514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:57.539746 13514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:57.541020 13514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:47:57.541081 13514 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:47:57.541100 13514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:57.541117 13514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:57.541133 13514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:47:57.541149 13514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:47:57.542836 13514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:43649
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:57.543148 13514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:57.543385 13514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:57.548828 13526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:57.548851 13530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:57.548863 13514 server_base.cc:1034] running on GCE node
W20260811 18:47:57.548827 13525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:57.549293 13514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:57.549641 13514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:57.550822 13514 hybrid_clock.cc:648] HybridClock initialized: now 1786474077550796 us; error 40 us; skew 500 ppm
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:57.553572 13514 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:47:57.554095 13514 webserver.cc:460] Webserver started at http://127.9.215.62:44357/ using document root <none> and password file <none>
I20260811 18:47:57.554320 13514 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:57.554368 13514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:57.554493 13514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:57.555686 13514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "5d7d0f300396497ab4bec6636ee1d4e0"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "93770e631733dadb816746259ea90a86"
server_key_iv: "86e9f4ffe7bddeba4034f2952d0579e9"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.556056 13514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "5d7d0f300396497ab4bec6636ee1d4e0"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "93770e631733dadb816746259ea90a86"
server_key_iv: "86e9f4ffe7bddeba4034f2952d0579e9"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.557561 13514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:47:57.558481 13540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.558877 13514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:57.558986 13514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "5d7d0f300396497ab4bec6636ee1d4e0"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "93770e631733dadb816746259ea90a86"
server_key_iv: "86e9f4ffe7bddeba4034f2952d0579e9"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.559078 13514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:57.572279 13514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:57.574627 13514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:57.574767 13514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:57.579846 13514 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:43649
I20260811 18:47:57.579914 13667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:43649 every 8 connection(s)
I20260811 18:47:57.580314 13514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:47:57.580958 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13514
I20260811 18:47:57.581058 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:47:57.581151 13668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:57.581251 10076 external_mini_cluster.cc:1428] Setting key b95d24493d19f0f1ab4d6c0fb48320ac
I20260811 18:47:57.584213 13668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Bootstrap starting.
I20260811 18:47:57.584860 13668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:57.585153 13668 log.cc:824] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:57.585938 13668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: No bootstrap required, opened a new log
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:57.588439 13668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43649 } }
I20260811 18:47:57.588615 13668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:57.588653 13668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d7d0f300396497ab4bec6636ee1d4e0, State: Initialized, Role: FOLLOWER
I20260811 18:47:57.588799 13668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43649 } }
I20260811 18:47:57.588891 13668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:47:57.588932 13668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:47:57.588984 13668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:57.589735 13668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43649 } }
I20260811 18:47:57.589860 13668 leader_election.cc:302] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d7d0f300396497ab4bec6636ee1d4e0; no voters:
I20260811 18:47:57.590073 13668 leader_election.cc:288] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:47:57.590127 13678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:57.590306 13678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [term 1 LEADER]: Becoming Leader. State: Replica: 5d7d0f300396497ab4bec6636ee1d4e0, State: Running, Role: LEADER
I20260811 18:47:57.590404 13668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:47:57.590411 13678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43649 } }
I20260811 18:47:57.590809 13678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d7d0f300396497ab4bec6636ee1d4e0. Latest consensus state: current_term: 1 leader_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43649 } } }
I20260811 18:47:57.590886 13678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:47:57.591037 13683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7d0f300396497ab4bec6636ee1d4e0" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43649 } } }
I20260811 18:47:57.591096 13683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:47:57.591214 13691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:47:57.591832 13691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:47:57.593055 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.581840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.9.215.62:43649)
0811 18:47:57.582051 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:57.582060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:57.582088 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:47:57.582585 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:57.582592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:57.582649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:57.582791 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:57.583437 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.584474 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.585350 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.585896 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.586020 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.588645 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:57.588650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:57.588650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:57.588665 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:57.590735 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.591221 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.591225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.591225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.591338 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.591625 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.591625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.591626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.591979 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:57.592630 (+ 651us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:57.592705 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:47:57.592849 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260811 18:47:57.593456 13691 catalog_manager.cc:1347] Generated new cluster ID: bf2044b2acfc4cf1b2a4690f30a7d25e
I20260811 18:47:57.593506 13691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:47:57.599905 13691 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:47:57.600701 13691 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:47:57.609859 13691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Generated new TSK 0
I20260811 18:47:57.610167 13691 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:47:57.647426 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43649
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:57.736330 13717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:57.736572 13717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:57.736614 13717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:57.737879 13717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:57.737931 13717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:57.737958 13717 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:47:57.737973 13717 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:47:57.738006 13717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:47:57.739692 13717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:57.739987 13717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:57.740231 13717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:57.745654 13732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:57.745647 13728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:57.745713 13730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:57.746047 13717 server_base.cc:1034] running on GCE node
I20260811 18:47:57.746318 13717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:57.746554 13717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:57.747736 13717 hybrid_clock.cc:648] HybridClock initialized: now 1786474077747716 us; error 31 us; skew 500 ppm
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:57.750352 13717 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:47:57.750828 13717 webserver.cc:460] Webserver started at http://127.9.215.1:37401/ using document root <none> and password file <none>
I20260811 18:47:57.751037 13717 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:57.751081 13717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:57.751175 13717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:57.752279 13717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "b32623c3009747ed8ffcb9dc9a43342f"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "20944fa22daebd866ab1a9d3f4fe7cc4"
server_key_iv: "f4738b93d248f8ca8a65b924ea8627cb"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.752599 13717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "b32623c3009747ed8ffcb9dc9a43342f"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "20944fa22daebd866ab1a9d3f4fe7cc4"
server_key_iv: "f4738b93d248f8ca8a65b924ea8627cb"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.754014 13717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:47:57.754916 13742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.755117 13717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:57.755234 13717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "b32623c3009747ed8ffcb9dc9a43342f"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "20944fa22daebd866ab1a9d3f4fe7cc4"
server_key_iv: "f4738b93d248f8ca8a65b924ea8627cb"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.755303 13717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:57.774503 13717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:57.776882 13717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:57.777012 13717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:57.777513 13717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:57.777551 13717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.777575 13717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:57.777587 13717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.784360 13717 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:38533
I20260811 18:47:57.784477 13962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:38533 every 8 connection(s)
I20260811 18:47:57.784761 13717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:57.788285 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.778152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:51105 (local address 127.9.215.62:43649)
0811 18:47:57.778224 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:57.778226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:57.778433 (+ 207us) server_negotiation.cc:407] Connection header received
0811 18:47:57.779037 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:57.779039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:57.779054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:57.779134 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:57.780078 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.780874 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.782424 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.782750 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.782832 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.785714 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:57.785716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:57.785717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:57.785725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:57.787057 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.787477 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.787478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.787479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.787539 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.787755 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.787755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.787755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.787862 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:57.787911 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:57.788138 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:47:57.788213 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:47:57.788481 13758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.778099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.9.215.1:51105)
0811 18:47:57.778290 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:57.778336 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:57.778910 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:57.779223 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:57.779229 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:57.779636 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:57.779947 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.779954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.780979 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.780980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.782295 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.782299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.782854 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.782855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.783035 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.783781 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:57.783792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:57.785586 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:57.787145 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.787153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.787155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.787388 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.787595 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.787595 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.787595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.787705 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.787900 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:57.787903 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:57.788029 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 18:47:57.788121 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:47:57.788255 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260811 18:47:57.789017 13964 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43649
I20260811 18:47:57.789125 13964 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:57.789340 13964 heartbeater.cc:511] Master 127.9.215.62:43649 requested a full tablet report, sending...
I20260811 18:47:57.789933 13573 ts_manager.cc:194] Registered new tserver with Master: b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533)
I20260811 18:47:57.790712 13573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:51105
I20260811 18:47:57.794021 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13717
I20260811 18:47:57.794127 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:47:57.794320 10076 external_mini_cluster.cc:1428] Setting key 0abe6588078497ac409b83f9ded456ee
I20260811 18:47:57.795873 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.790337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:47151 (local address 127.9.215.62:43649)
0811 18:47:57.790383 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:57.790385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:57.790456 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:47:57.790517 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:57.790517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:57.790527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:57.790604 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:57.790936 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.791542 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.792259 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.792480 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.792543 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.793480 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:57.793482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:57.793482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:57.793489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:57.794825 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.795170 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.795171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.795172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.795228 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.795410 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.795410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.795410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.795533 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:57.795587 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:57.795743 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 18:47:57.795811 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:47:57.795930 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.790220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.9.215.1:47151)
0811 18:47:57.790399 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:57.790410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:57.790472 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:57.790631 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:57.790632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:57.790796 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:57.790866 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.790868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.791637 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.791637 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.792165 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.792167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.792540 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.792541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.792623 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.793078 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:57.793083 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:57.793398 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:57.794893 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.794894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.794895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.795108 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.795292 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.795293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.795293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.795357 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.795589 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:57.795590 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:57.795653 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 18:47:57.795719 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:47:57.795831 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:47:57.823375 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43649
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:57.910024 13975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:57.910287 13975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:57.910316 13975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:57.911573 13975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:57.911613 13975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:57.911633 13975 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:47:57.911643 13975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:47:57.911659 13975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:47:57.913169 13975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:57.913391 13975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:57.913584 13975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:57.918619 13995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:57.918641 13984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:57.918658 13987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:57.918927 13975 server_base.cc:1034] running on GCE node
I20260811 18:47:57.919198 13975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:57.919483 13975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:57.920672 13975 hybrid_clock.cc:648] HybridClock initialized: now 1786474077920657 us; error 40 us; skew 500 ppm
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:57.923300 13975 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:47:57.923861 13975 webserver.cc:460] Webserver started at http://127.9.215.2:44431/ using document root <none> and password file <none>
I20260811 18:47:57.924098 13975 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:57.924152 13975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:57.924283 13975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:57.925374 13975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "ee236172dc004dcdb84081776247f5de"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "0cbb5174ddc4586d6c81da32f997240c"
server_key_iv: "09620e1547f1f0af733b2b19f35b5988"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.925757 13975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "ee236172dc004dcdb84081776247f5de"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "0cbb5174ddc4586d6c81da32f997240c"
server_key_iv: "09620e1547f1f0af733b2b19f35b5988"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.927126 13975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:47:57.927932 14010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.928095 13975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.928188 13975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "ee236172dc004dcdb84081776247f5de"
format_stamp: "Formatted at 2026-08-11 18:47:57 on dist-test-slave-pww0"
server_key: "0cbb5174ddc4586d6c81da32f997240c"
server_key_iv: "09620e1547f1f0af733b2b19f35b5988"
server_key_version: "encryptionkey@0"
I20260811 18:47:57.928253 13975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:57.940210 13975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:57.942553 13975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:57.942713 13975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:57.943295 13975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:57.943351 13975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:57.943382 13975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:57.943398 13975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:47:57 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:57.950457 13975 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:37637
I20260811 18:47:57.950815 13975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:47:57.950814 14234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:37637 every 8 connection(s)
I20260811 18:47:57.953048 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.943752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:45519 (local address 127.9.215.62:43649)
0811 18:47:57.943817 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:57.943819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:57.944069 (+ 250us) server_negotiation.cc:407] Connection header received
0811 18:47:57.944749 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:57.944751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:57.944765 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:57.944828 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:57.945761 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.946339 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.947791 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.948008 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.948066 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.950277 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:57.950279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:57.950280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:57.950288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:57.951770 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.952142 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.952144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.952144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.952201 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.952416 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.952417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.952418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.952534 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:57.952591 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:57.952870 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 18:47:57.952957 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:47:57.953245 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.943731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.9.215.2:45519)
0811 18:47:57.943920 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:57.943972 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:57.944634 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:57.944923 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:57.944928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:57.945327 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:57.945645 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.945653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.946425 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.946426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.947712 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.947715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.948098 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.948100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.948234 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.948728 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:57.948737 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:57.950173 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:57.951843 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.951850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.951853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.952070 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.952254 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.952255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.952255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.952362 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.952589 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:57.952597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:57.952747 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 18:47:57.952839 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:47:57.953010 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260811 18:47:57.953730 14235 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43649
I20260811 18:47:57.953815 14235 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:57.954012 14235 heartbeater.cc:511] Master 127.9.215.62:43649 requested a full tablet report, sending...
I20260811 18:47:57.954396 13577 ts_manager.cc:194] Registered new tserver with Master: ee236172dc004dcdb84081776247f5de (127.9.215.2:37637)
I20260811 18:47:57.954901 13577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:45519
I20260811 18:47:57.959719 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13975
I20260811 18:47:57.959939 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:47:57.960152 10076 external_mini_cluster.cc:1428] Setting key 26917b5ef7ee724746abf018d3bd0e26
I20260811 18:47:57.961041 14244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.954799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.9.215.2:34349)
0811 18:47:57.954948 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:57.954958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:57.955023 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:57.955147 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:57.955147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:57.955324 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:57.955394 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.955397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.956156 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.956156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.956663 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:57.956664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.957027 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.957028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:57.957099 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.957587 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:57.957591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:57.957898 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:57.959569 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.959571 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.959572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.959801 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.960257 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:57.960258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:57.960259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:57.960338 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:57.960660 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:57.960661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:57.960750 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 18:47:57.960823 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 18:47:57.960952 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260811 18:47:57.961050 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:57.954861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:34349 (local address 127.9.215.62:43649)
0811 18:47:57.954901 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:57.954903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:57.955010 (+ 107us) server_negotiation.cc:407] Connection header received
0811 18:47:57.955080 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:57.955081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:57.955090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:57.955131 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:57.955595 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.956069 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.956741 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:57.956935 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:57.956989 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:57.957994 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:57.957995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:57.957995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:57.958002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:57.959485 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.960089 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.960091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.960091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.960162 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:57.960398 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:57.960399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:57.960400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:57.960588 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:57.960654 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:57.960853 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 18:47:57.960959 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:47:57.988466 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43649
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:47:58.073625 14250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:47:58.073871 14250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:47:58.073899 14250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:47:58.075083 14250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:47:58.075131 14250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:47:58.075152 14250 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:47:58.075162 14250 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:47:58.075178 14250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:47:58.076783 14250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:47:58.077008 14250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:47:58.077207 14250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:47:58.082581 14260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:58.082804 14250 server_base.cc:1034] running on GCE node
W20260811 18:47:58.082999 14267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:47:58.083101 14262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:47:58.083724 14250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:47:58.083966 14250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:47:58.085110 14250 hybrid_clock.cc:648] HybridClock initialized: now 1786474078085082 us; error 46 us; skew 500 ppm
Aug 11 18:47:58 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474078, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:47:58.087755 14250 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:47:58.088297 14250 webserver.cc:460] Webserver started at http://127.9.215.3:44697/ using document root <none> and password file <none>
I20260811 18:47:58.088533 14250 fs_manager.cc:359] Metadata directory not provided
I20260811 18:47:58.088579 14250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:47:58.088677 14250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:47:58.089789 14250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "9f0ecd24231242ab8e296baa444cff3e"
format_stamp: "Formatted at 2026-08-11 18:47:58 on dist-test-slave-pww0"
server_key: "0681447fa947757202669c3c588d5871"
server_key_iv: "951b62ebfee5400510667cc495e08569"
server_key_version: "encryptionkey@0"
I20260811 18:47:58.090144 14250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "9f0ecd24231242ab8e296baa444cff3e"
format_stamp: "Formatted at 2026-08-11 18:47:58 on dist-test-slave-pww0"
server_key: "0681447fa947757202669c3c588d5871"
server_key_iv: "951b62ebfee5400510667cc495e08569"
server_key_version: "encryptionkey@0"
I20260811 18:47:58.091578 14250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:47:58.092379 14281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:58.092540 14250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:58.092639 14250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "9f0ecd24231242ab8e296baa444cff3e"
format_stamp: "Formatted at 2026-08-11 18:47:58 on dist-test-slave-pww0"
server_key: "0681447fa947757202669c3c588d5871"
server_key_iv: "951b62ebfee5400510667cc495e08569"
server_key_version: "encryptionkey@0"
I20260811 18:47:58.092722 14250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:47:58.104362 14250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:47:58.106483 14250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:47:58.106611 14250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:47:58.107275 14250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:47:58.107331 14250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:58.107362 14250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:47:58.107378 14250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:47:58 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474078, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:58.114887 14250 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:34307
I20260811 18:47:58.114998 14508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:34307 every 8 connection(s)
I20260811 18:47:58.115491 14250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:47:58.117494 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.107809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:48985 (local address 127.9.215.62:43649)
0811 18:47:58.107881 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.107884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.108095 (+ 211us) server_negotiation.cc:407] Connection header received
0811 18:47:58.108690 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.108692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.108707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.108792 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:58.109561 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.110391 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.111641 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.111941 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.112010 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.114542 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:58.114544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:58.114544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:58.114552 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:58.116220 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.116692 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.116694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.116694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.116756 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.116955 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.116955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.116956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.117062 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:58.117110 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.117342 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.117427 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:47:58.117662 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.107769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.9.215.3:48985)
0811 18:47:58.107958 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.108002 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.108554 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.108884 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.108888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.109199 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:58.109449 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.109456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.110574 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.110575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.111447 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.111450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.112050 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.112052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.112224 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.112807 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:58.112817 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:58.114414 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:58.116297 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:58.116301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:58.116302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:58.116578 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:58.116812 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:58.116812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:58.116813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:58.116912 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:58.117113 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:58.117116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:58.117237 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.117320 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.117453 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260811 18:47:58.118220 14509 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43649
I20260811 18:47:58.118307 14509 heartbeater.cc:464] Registering TS with master...
I20260811 18:47:58.118490 14509 heartbeater.cc:511] Master 127.9.215.62:43649 requested a full tablet report, sending...
I20260811 18:47:58.118888 13577 ts_manager.cc:194] Registered new tserver with Master: 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:47:58.119292 13577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:48985
I20260811 18:47:58.124925 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.119322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.9.215.3:42211)
0811 18:47:58.119639 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.119649 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.119720 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.119830 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.119831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.120035 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:58.120112 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.120114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.120758 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.120759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.121310 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.121312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.121733 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.121734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.121802 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.122276 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:58.122282 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:58.122621 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:58.124017 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:58.124018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:58.124018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:58.124204 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:58.124362 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:58.124363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:58.124363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:58.124410 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:58.124649 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:58.124650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:58.124698 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.124750 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.124839 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":85,"threads_started":1}
I20260811 18:47:58.124963 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.119375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:42211 (local address 127.9.215.62:43649)
0811 18:47:58.119415 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.119417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.119696 (+ 279us) server_negotiation.cc:407] Connection header received
0811 18:47:58.119773 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.119773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.119782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.119826 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:58.120182 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.120650 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.121414 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.121633 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.121691 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.122699 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:58.122700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:58.122701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:58.122707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:58.123946 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.124268 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.124269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.124269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.124319 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.124454 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.124455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.124455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.124572 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:58.124649 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.124795 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.124874 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 18:47:58.124979 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 14250
I20260811 18:47:58.125049 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:47:58.125208 10076 external_mini_cluster.cc:1428] Setting key 2cab6e55836d5f58284cb61672a7725b
I20260811 18:47:58.126108 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:47:58 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:47:58 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:58.152068 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.144290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.9.215.62:43649)
0811 18:47:58.144364 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.144366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.144376 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:47:58.144402 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.144403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.144412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.144535 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:58.145203 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.145790 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.146566 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.146773 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.146832 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.149366 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:58.149368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:58.149369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:58.149378 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:58.150855 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.151234 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.151236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.151236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.151328 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.151527 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.151527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.151528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.151661 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:58.151758 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.151874 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.151965 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:47:58.154054 13577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:47:58.155891 13577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:58.162997 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.158420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38533 (local address 127.0.0.1:47674)
0811 18:47:58.158642 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.158694 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.158784 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.159167 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.159170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.159173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.159255 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.159257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.160458 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.160459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.161627 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.161631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.162657 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.162658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.162752 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.162768 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.162838 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.162900 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260811 18:47:58.163156 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.158770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.9.215.2:37637)
0811 18:47:58.159006 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.159012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.159024 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:47:58.159067 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.159069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.159084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.159371 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.159445 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.160911 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.162126 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.162776 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.162842 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.162910 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.163004 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.163044 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":101,"threads_started":1}
I20260811 18:47:58.163479 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.158437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:37637 (local address 127.0.0.1:36524)
0811 18:47:58.158818 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.158828 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.158869 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.159221 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.159222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.159224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.159290 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.159292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.161012 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.161013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.162010 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.162013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.162852 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.162853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.162919 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.162923 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.162987 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.163030 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":134,"threads_started":1}
I20260811 18:47:58.163575 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.158666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:34307 (local address 127.0.0.1:53498)
0811 18:47:58.159141 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.159152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.159217 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.159825 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.159827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.159830 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.159904 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.159905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.161281 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.161282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.162370 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.162373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.163309 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.163311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.163383 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.163387 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.163437 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.163505 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":117,"threads_started":1}
I20260811 18:47:58.163632 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.158929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.9.215.3:34307)
0811 18:47:58.159192 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.159196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.159208 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:47:58.159481 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.159551 (+ 70us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.159599 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.159786 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.159990 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.161203 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.162464 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.163239 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.163308 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.163366 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.163490 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.163529 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260811 18:47:58.164050 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.158531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.9.215.1:38533)
0811 18:47:58.158722 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.158727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.158757 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:47:58.158944 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.158947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.158963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.159017 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.159318 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.160383 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.161758 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.162574 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.162639 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.162767 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.163697 (+ 930us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.163770 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260811 18:47:58.164362 14107 tablet_service.cc:1467] Processing CreateTablet for tablet abea0ad929774957a23530e7e6fce015 (DEFAULT_TABLE table=test-table [id=b5d25edf6526497da0aa9dc42111288c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:58.164482 14379 tablet_service.cc:1467] Processing CreateTablet for tablet abea0ad929774957a23530e7e6fce015 (DEFAULT_TABLE table=test-table [id=b5d25edf6526497da0aa9dc42111288c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:58.164774 14379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abea0ad929774957a23530e7e6fce015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.164669 13844 tablet_service.cc:1467] Processing CreateTablet for tablet abea0ad929774957a23530e7e6fce015 (DEFAULT_TABLE table=test-table [id=b5d25edf6526497da0aa9dc42111288c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:58.164781 14107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abea0ad929774957a23530e7e6fce015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.164996 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abea0ad929774957a23530e7e6fce015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.167608 14552 tablet_bootstrap.cc:492] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Bootstrap starting.
I20260811 18:47:58.168681 14554 tablet_bootstrap.cc:492] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Bootstrap starting.
I20260811 18:47:58.168823 14552 tablet_bootstrap.cc:654] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.169338 14552 log.cc:824] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:58.169584 14554 tablet_bootstrap.cc:654] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.169778 14556 tablet_bootstrap.cc:492] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Bootstrap starting.
I20260811 18:47:58.169930 14554 log.cc:824] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:58.170214 14552 tablet_bootstrap.cc:492] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: No bootstrap required, opened a new log
I20260811 18:47:58.170315 14552 ts_tablet_manager.cc:1397] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:47:58.170455 14556 tablet_bootstrap.cc:654] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.170759 14554 tablet_bootstrap.cc:492] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: No bootstrap required, opened a new log
I20260811 18:47:58.170786 14556 log.cc:824] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Log is configured to *not* fsync() on all Append() calls
I20260811 18:47:58.170833 14554 ts_tablet_manager.cc:1397] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:47:58.171664 14556 tablet_bootstrap.cc:492] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: No bootstrap required, opened a new log
I20260811 18:47:58.171746 14556 ts_tablet_manager.cc:1397] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:47:58.172179 14552 raft_consensus.cc:348] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.172495 14552 raft_consensus.cc:374] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.172549 14552 raft_consensus.cc:729] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.172672 14552 consensus_queue.cc:260] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.172972 14552 ts_tablet_manager.cc:1428] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:47:58.173115 14509 heartbeater.cc:503] Master 127.9.215.62:43649 was elected leader, sending a full tablet report...
I20260811 18:47:58.173976 14554 raft_consensus.cc:348] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.174166 14554 raft_consensus.cc:374] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.174213 14554 raft_consensus.cc:729] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.174329 14554 consensus_queue.cc:260] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.174692 14554 ts_tablet_manager.cc:1428] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 18:47:58.174774 13964 heartbeater.cc:503] Master 127.9.215.62:43649 was elected leader, sending a full tablet report...
I20260811 18:47:58.174665 14556 raft_consensus.cc:348] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.174901 14556 raft_consensus.cc:374] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.174930 14556 raft_consensus.cc:729] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee236172dc004dcdb84081776247f5de, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.175031 14556 consensus_queue.cc:260] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.175344 14556 ts_tablet_manager.cc:1428] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:47:58.175424 14235 heartbeater.cc:503] Master 127.9.215.62:43649 was elected leader, sending a full tablet report...
W20260811 18:47:58.201643 14236 tablet.cc:2369] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:47:58.285542 13966 tablet.cc:2369] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:47:58.346880 14564 raft_consensus.cc:482] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:58.347020 14564 raft_consensus.cc:504] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.347429 14564 leader_election.cc:288] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee236172dc004dcdb84081776247f5de (127.9.215.2:37637), 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:47:58.350548 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.347638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:37637 (local address 127.9.215.1:50229)
0811 18:47:58.347840 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.347851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.347921 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.348046 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.348047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.348049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.348123 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.348125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.348866 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.348867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.349706 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.349709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.350254 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.350256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.350348 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.350352 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.350399 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.350450 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":4,"thread_start_us":77,"threads_started":1}
I20260811 18:47:58.350541 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.347726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:50229 (local address 127.9.215.2:37637)
0811 18:47:58.347781 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.347783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.347916 (+ 133us) server_negotiation.cc:407] Connection header received
0811 18:47:58.347978 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.347979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.347991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.348037 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.348180 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.348761 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.349807 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.350179 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.350243 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.350254 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.350410 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.350451 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:47:58.350692 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.347756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:57173 (local address 127.9.215.3:34307)
0811 18:47:58.347808 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.347810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.347976 (+ 166us) server_negotiation.cc:407] Connection header received
0811 18:47:58.348040 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.348041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.348052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.348092 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.348247 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.348769 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.349924 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.350334 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.350415 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.350429 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.350563 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.350606 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:47:58.350688 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.347670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:34307 (local address 127.9.215.1:57173)
0811 18:47:58.347901 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.347911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.347974 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.348118 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.348119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.348122 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.348188 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.348190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.348848 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.348849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.349765 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.349768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.350418 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.350419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.350501 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.350504 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.350550 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.350596 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":27,"thread_start_us":64,"threads_started":1}
I20260811 18:47:58.350943 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abea0ad929774957a23530e7e6fce015" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de" is_pre_election: true
I20260811 18:47:58.351030 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abea0ad929774957a23530e7e6fce015" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0ecd24231242ab8e296baa444cff3e" is_pre_election: true
I20260811 18:47:58.351109 14144 raft_consensus.cc:2454] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 0.
I20260811 18:47:58.351186 14422 raft_consensus.cc:2454] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 0.
I20260811 18:47:58.351379 13745 leader_election.cc:302] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32623c3009747ed8ffcb9dc9a43342f, ee236172dc004dcdb84081776247f5de; no voters:
I20260811 18:47:58.351554 14564 raft_consensus.cc:2781] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:58.351639 14564 raft_consensus.cc:482] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:58.351672 14564 raft_consensus.cc:3037] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.352421 14564 raft_consensus.cc:504] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.352560 14564 leader_election.cc:288] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 election: Requested vote from peers ee236172dc004dcdb84081776247f5de (127.9.215.2:37637), 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:47:58.352753 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abea0ad929774957a23530e7e6fce015" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de"
I20260811 18:47:58.352766 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abea0ad929774957a23530e7e6fce015" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0ecd24231242ab8e296baa444cff3e"
I20260811 18:47:58.352840 14144 raft_consensus.cc:3037] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.352842 14422 raft_consensus.cc:3037] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.353660 14422 raft_consensus.cc:2454] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 1.
I20260811 18:47:58.353688 14144 raft_consensus.cc:2454] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 1.
I20260811 18:47:58.353888 13745 leader_election.cc:302] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, b32623c3009747ed8ffcb9dc9a43342f, ee236172dc004dcdb84081776247f5de; no voters:
I20260811 18:47:58.354000 14564 raft_consensus.cc:2781] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:58.354125 14564 raft_consensus.cc:686] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 1 LEADER]: Becoming Leader. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Running, Role: LEADER
I20260811 18:47:58.354229 14564 consensus_queue.cc:237] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.355134 13573 catalog_manager.cc:5495] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f reported cstate change: term changed from 0 to 1, leader changed from <none> to b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1). New cstate: current_term: 1 leader_uuid: "b32623c3009747ed8ffcb9dc9a43342f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } health_report { overall_health: UNKNOWN } } }
W20260811 18:47:58.366130 14513 tablet.cc:2369] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:47:58 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:47:58.418605 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.408384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.9.215.62:43649)
0811 18:47:58.408468 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.408471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.408598 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:47:58.408668 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.408669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.408679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.408726 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:47:58.409245 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.409735 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.410426 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.410605 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.410658 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.416078 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:47:58.416079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:47:58.416080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:47:58.416088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:47:58.417413 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.417833 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.417834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.417835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.417888 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:47:58.418045 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:47:58.418045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:47:58.418046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:47:58.418159 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:47:58.418216 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.418362 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.418469 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:47:58.418681 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.408308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43649 (local address 127.0.0.1:53406)
0811 18:47:58.408514 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.408524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.408607 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.408740 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.408741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.409103 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:47:58.409156 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.409159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.409810 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.409811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.410348 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.410349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.410666 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.410667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.410718 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.414419 (+ 3701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:47:58.414426 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:47:58.415962 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:47:58.417511 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:58.417523 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:58.417526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:58.417763 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:58.417942 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:47:58.417943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:47:58.417943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:47:58.417996 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:47:58.418214 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:47:58.418225 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:47:58.418277 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.418333 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.418542 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":94,"thread_start_us":100,"threads_started":1}
I20260811 18:47:58.420933 13573 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:53406:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:47:58.421177 13573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:58.422757 13844 tablet_service.cc:1467] Processing CreateTablet for tablet 2acd048d74ba46a2b728225da53c5e3e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cf651976a9d481b8ede89947ed2dd94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:47:58.422802 14379 tablet_service.cc:1467] Processing CreateTablet for tablet 2acd048d74ba46a2b728225da53c5e3e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cf651976a9d481b8ede89947ed2dd94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:47:58.422796 14107 tablet_service.cc:1467] Processing CreateTablet for tablet 2acd048d74ba46a2b728225da53c5e3e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3cf651976a9d481b8ede89947ed2dd94]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:47:58.422967 14379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2acd048d74ba46a2b728225da53c5e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.422967 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2acd048d74ba46a2b728225da53c5e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.422966 14107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2acd048d74ba46a2b728225da53c5e3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.424345 14556 tablet_bootstrap.cc:492] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: Bootstrap starting.
I20260811 18:47:58.424587 14552 tablet_bootstrap.cc:492] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: Bootstrap starting.
I20260811 18:47:58.425129 14556 tablet_bootstrap.cc:654] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.425172 14552 tablet_bootstrap.cc:654] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.425576 14554 tablet_bootstrap.cc:492] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: Bootstrap starting.
I20260811 18:47:58.425957 14556 tablet_bootstrap.cc:492] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: No bootstrap required, opened a new log
I20260811 18:47:58.425992 14554 tablet_bootstrap.cc:654] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.426019 14556 ts_tablet_manager.cc:1397] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:47:58.426191 14556 raft_consensus.cc:348] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.426278 14556 raft_consensus.cc:374] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.426298 14556 raft_consensus.cc:729] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee236172dc004dcdb84081776247f5de, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.426378 14556 consensus_queue.cc:260] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.426501 14556 ts_tablet_manager.cc:1428] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:58.426573 14552 tablet_bootstrap.cc:492] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: No bootstrap required, opened a new log
I20260811 18:47:58.426622 14552 ts_tablet_manager.cc:1397] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:47:58.426759 14552 raft_consensus.cc:348] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.426821 14552 raft_consensus.cc:374] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.426833 14552 raft_consensus.cc:729] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.426874 14552 consensus_queue.cc:260] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.426957 14552 ts_tablet_manager.cc:1428] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:58.427109 14565 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.427068 14562 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.427184 14565 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.427187 14562 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.428103 14554 tablet_bootstrap.cc:492] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: No bootstrap required, opened a new log
I20260811 18:47:58.428169 14554 ts_tablet_manager.cc:1397] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:47:58.428495 14554 raft_consensus.cc:348] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.428565 14554 raft_consensus.cc:374] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.428583 14554 raft_consensus.cc:729] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.428628 14554 consensus_queue.cc:260] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.428725 14554 ts_tablet_manager.cc:1428] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:47:58.428848 14564 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.428911 14564 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.553277 14565 raft_consensus.cc:482] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:58.553397 14565 raft_consensus.cc:504] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.553793 14565 leader_election.cc:288] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533), 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:47:58.556898 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.554156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:57235 (local address 127.9.215.1:38533)
0811 18:47:58.554277 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.554280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.554295 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:47:58.554330 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.554331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.554344 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.554426 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.554590 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.555115 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.556202 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.556566 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.556629 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.556641 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.556783 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.556822 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 18:47:58.556938 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.554386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:46639 (local address 127.9.215.3:34307)
0811 18:47:58.554434 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.554436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.554444 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:47:58.554472 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.554473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.554479 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.554523 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.554699 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.555278 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.556252 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.556617 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.556679 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.556690 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.556827 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.556866 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:47:58.556954 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.554228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:34307 (local address 127.9.215.2:46639)
0811 18:47:58.554337 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.554346 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.554382 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.554541 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.554542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.554544 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.554607 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.554609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.555386 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.555387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.556167 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.556170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.556691 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.556692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.556768 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.556772 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.556817 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.556858 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260811 18:47:58.556958 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.554031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38533 (local address 127.9.215.2:57235)
0811 18:47:58.554181 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.554194 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.554254 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.554440 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.554442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.554445 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.554523 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.554525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.555189 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.555190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.556101 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.556104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.556640 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.556641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.556720 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.556724 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.556770 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.556817 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":21,"thread_start_us":76,"threads_started":1}
I20260811 18:47:58.557245 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd048d74ba46a2b728225da53c5e3e" candidate_uuid: "ee236172dc004dcdb84081776247f5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0ecd24231242ab8e296baa444cff3e" is_pre_election: true
I20260811 18:47:58.557286 13886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd048d74ba46a2b728225da53c5e3e" candidate_uuid: "ee236172dc004dcdb84081776247f5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32623c3009747ed8ffcb9dc9a43342f" is_pre_election: true
I20260811 18:47:58.557354 14422 raft_consensus.cc:2454] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee236172dc004dcdb84081776247f5de in term 0.
I20260811 18:47:58.557389 13886 raft_consensus.cc:2454] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee236172dc004dcdb84081776247f5de in term 0.
I20260811 18:47:58.557541 14017 leader_election.cc:302] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, ee236172dc004dcdb84081776247f5de; no voters:
I20260811 18:47:58.557693 14565 raft_consensus.cc:2781] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:58.557739 14565 raft_consensus.cc:482] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:58.557765 14565 raft_consensus.cc:3037] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.558502 14565 raft_consensus.cc:504] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.558625 14565 leader_election.cc:288] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [CANDIDATE]: Term 1 election: Requested vote from peers b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533), 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:47:58.558827 13886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd048d74ba46a2b728225da53c5e3e" candidate_uuid: "ee236172dc004dcdb84081776247f5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32623c3009747ed8ffcb9dc9a43342f"
I20260811 18:47:58.558885 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2acd048d74ba46a2b728225da53c5e3e" candidate_uuid: "ee236172dc004dcdb84081776247f5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0ecd24231242ab8e296baa444cff3e"
I20260811 18:47:58.558930 13886 raft_consensus.cc:3037] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.558965 14422 raft_consensus.cc:3037] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.559823 13886 raft_consensus.cc:2454] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee236172dc004dcdb84081776247f5de in term 1.
I20260811 18:47:58.559823 14422 raft_consensus.cc:2454] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee236172dc004dcdb84081776247f5de in term 1.
I20260811 18:47:58.560010 14017 leader_election.cc:302] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, ee236172dc004dcdb84081776247f5de; no voters:
I20260811 18:47:58.560112 14565 raft_consensus.cc:2781] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:58.560236 14565 raft_consensus.cc:686] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [term 1 LEADER]: Becoming Leader. State: Replica: ee236172dc004dcdb84081776247f5de, State: Running, Role: LEADER
I20260811 18:47:58.560333 14565 consensus_queue.cc:237] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.560822 14595 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: TxnStatusTablet state changed. Reason: New leader ee236172dc004dcdb84081776247f5de. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.560910 14595 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:47:58.561209 14603 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:47:58.561393 14603 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:47:58.561295 13571 catalog_manager.cc:5495] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de reported cstate change: term changed from 0 to 1, leader changed from <none> to ee236172dc004dcdb84081776247f5de (127.9.215.2). New cstate: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } health_report { overall_health: UNKNOWN } } }
I20260811 18:47:58.575275 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.572613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.9.215.2:37637)
0811 18:47:58.572672 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.572674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.572682 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:47:58.572714 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.572715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.572732 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.572787 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.572933 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.573407 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.574510 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.574727 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.574784 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.574971 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.575074 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.575132 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.575185 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.575209 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:47:58.575322 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.572364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:37637 (local address 127.0.0.1:36540)
0811 18:47:58.572440 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.572454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.572511 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.572813 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.572814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.572815 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:58.572874 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.572876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.573468 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.573469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.574410 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.574414 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.574794 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.574795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.574861 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.574884 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.575127 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.575133 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.575182 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.575232 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:47:58.577349 13886 raft_consensus.cc:1261] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Refusing update from remote peer ee236172dc004dcdb84081776247f5de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:58.577351 14422 raft_consensus.cc:1261] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Refusing update from remote peer ee236172dc004dcdb84081776247f5de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:58.577708 14565 consensus_queue.cc:1035] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:58.577841 14595 consensus_queue.cc:1035] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:58.579082 14562 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: TxnStatusTablet state changed. Reason: New leader ee236172dc004dcdb84081776247f5de. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.579200 14562 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.579491 14564 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: TxnStatusTablet state changed. Reason: New leader ee236172dc004dcdb84081776247f5de. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.579588 14564 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.580760 14595 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.580853 14595 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:47:58.581557 14564 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.581650 14564 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P b32623c3009747ed8ffcb9dc9a43342f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.582156 14606 mvcc.cc:205] Tried to move back new op lower bound from 7317397825849896960 to 7317397825783930880. Current Snapshot: MvccSnapshot[applied={T|T < 7317397825849896960}]
I20260811 18:47:58.582443 14565 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.582527 14565 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P ee236172dc004dcdb84081776247f5de: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:47:58.582973 14608 mvcc.cc:205] Tried to move back new op lower bound from 7317397825849896960 to 7317397825783930880. Current Snapshot: MvccSnapshot[applied={T|T < 7317397825849896960}]
I20260811 18:47:58.583173 14562 tablet_replica.cc:439] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee236172dc004dcdb84081776247f5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } } }
I20260811 18:47:58.583263 14562 tablet_replica.cc:442] T 2acd048d74ba46a2b728225da53c5e3e P 9f0ecd24231242ab8e296baa444cff3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:47:58.583411 14610 mvcc.cc:205] Tried to move back new op lower bound from 7317397825849896960 to 7317397825783930880. Current Snapshot: MvccSnapshot[applied={T|T < 7317397825849896960}]
I20260811 18:47:58.587252 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.584960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.9.215.1:38533)
0811 18:47:58.585025 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.585027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.585037 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:47:58.585071 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.585072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.585091 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.585153 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.585272 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.585779 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.586523 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.586734 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.586788 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.586937 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.587043 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.587113 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.587155 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.587180 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:47:58.593075 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.589997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:43323 (local address 127.9.215.2:37637)
0811 18:47:58.590038 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.590040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.590202 (+ 162us) server_negotiation.cc:407] Connection header received
0811 18:47:58.590250 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.590251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.590270 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.590315 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.590504 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.591122 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.592375 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.592604 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.592667 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.592867 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.592894 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.592925 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.592977 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.592992 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 18:47:58.593128 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.589943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:37637 (local address 127.9.215.1:43323)
0811 18:47:58.590135 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.590146 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.590206 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.590320 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.590321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.590323 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:58.590397 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.590398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.591196 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.591197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.592271 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.592274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.592675 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.592676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.592749 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.592770 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.592929 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.592930 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.592974 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.593019 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260811 18:47:58.595602 14422 raft_consensus.cc:1261] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Refusing update from remote peer b32623c3009747ed8ffcb9dc9a43342f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:58.595796 14144 raft_consensus.cc:1261] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Refusing update from remote peer b32623c3009747ed8ffcb9dc9a43342f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:58.596122 14564 consensus_queue.cc:1035] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:58.596276 14570 consensus_queue.cc:1035] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:58.606447 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.603088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38533 (local address 127.9.215.2:46443)
0811 18:47:58.603296 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.603309 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.603370 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.603603 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.603604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.603606 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:58.603686 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.603687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.604620 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.604622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.605540 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.605544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.605976 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.605977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.606047 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.606069 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.606239 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.606241 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.606282 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.606331 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260811 18:47:58.606420 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.603158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:46443 (local address 127.9.215.1:38533)
0811 18:47:58.603216 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.603218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.603376 (+ 158us) server_negotiation.cc:407] Connection header received
0811 18:47:58.603427 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.603428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.603443 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.603534 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.603753 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.604476 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.605618 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.605897 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.605958 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.606143 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.606185 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.606232 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.606284 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.606334 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:47:58.617108 13571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:47:58.617241 13571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:47:58.618326 13571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Sending DeleteTablet for 3 replicas of tablet abea0ad929774957a23530e7e6fce015
I20260811 18:47:58.618705 13844 tablet_service.cc:1514] Processing DeleteTablet for tablet abea0ad929774957a23530e7e6fce015 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:58 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:47674
I20260811 18:47:58.618829 14379 tablet_service.cc:1514] Processing DeleteTablet for tablet abea0ad929774957a23530e7e6fce015 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:58 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:53498
I20260811 18:47:58.619071 14648 tablet_replica.cc:330] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: stopping tablet replica
I20260811 18:47:58.619194 14648 raft_consensus.cc:2229] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:47:58.619225 14107 tablet_service.cc:1514] Processing DeleteTablet for tablet abea0ad929774957a23530e7e6fce015 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:47:58 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36524
I20260811 18:47:58.619321 14648 raft_consensus.cc:2258] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:58.619830 14648 ts_tablet_manager.cc:1905] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:58.619905 13571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:47:58.620069 13571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:47:58.621305 14651 tablet_replica.cc:330] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: stopping tablet replica
I20260811 18:47:58.621527 14651 raft_consensus.cc:2229] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:47:58.621722 14648 ts_tablet_manager.cc:1918] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:47:58.621743 13844 tablet_service.cc:1467] Processing CreateTablet for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd (DEFAULT_TABLE table=test-table [id=ea63baa04101436a87df180e042f109b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:58.621745 14107 tablet_service.cc:1467] Processing CreateTablet for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd (DEFAULT_TABLE table=test-table [id=ea63baa04101436a87df180e042f109b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:58.621814 14648 log.cc:1196] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/abea0ad929774957a23530e7e6fce015
I20260811 18:47:58.621862 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.621861 14107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.622088 14648 ts_tablet_manager.cc:1939] T abea0ad929774957a23530e7e6fce015 P b32623c3009747ed8ffcb9dc9a43342f: Deleting consensus metadata
I20260811 18:47:58.622522 13547 catalog_manager.cc:4841] TS b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533): tablet abea0ad929774957a23530e7e6fce015 (table test-table [id=b5d25edf6526497da0aa9dc42111288c]) successfully deleted
I20260811 18:47:58.623075 14554 tablet_bootstrap.cc:492] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Bootstrap starting.
I20260811 18:47:58.623155 14379 tablet_service.cc:1467] Processing CreateTablet for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd (DEFAULT_TABLE table=test-table [id=ea63baa04101436a87df180e042f109b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:47:58.623270 14379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:47:58.623319 14556 tablet_bootstrap.cc:492] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Bootstrap starting.
I20260811 18:47:58.623693 14554 tablet_bootstrap.cc:654] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.623742 14556 tablet_bootstrap.cc:654] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.624325 14554 tablet_bootstrap.cc:492] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: No bootstrap required, opened a new log
I20260811 18:47:58.624368 14554 ts_tablet_manager.cc:1397] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:47:58.624505 14554 raft_consensus.cc:348] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.624583 14554 raft_consensus.cc:374] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.624603 14554 raft_consensus.cc:729] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.624645 14554 consensus_queue.cc:260] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.624651 14552 tablet_bootstrap.cc:492] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Bootstrap starting.
I20260811 18:47:58.624725 14554 ts_tablet_manager.cc:1428] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 18:47:58.624935 14651 raft_consensus.cc:2258] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:58.625056 14552 tablet_bootstrap.cc:654] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Neither blocks nor log segments found. Creating new log.
I20260811 18:47:58.625752 14552 tablet_bootstrap.cc:492] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: No bootstrap required, opened a new log
I20260811 18:47:58.625818 14552 ts_tablet_manager.cc:1397] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:47:58.625933 14552 raft_consensus.cc:348] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.625993 14552 raft_consensus.cc:374] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.626012 14552 raft_consensus.cc:729] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.626051 14552 consensus_queue.cc:260] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.626128 14552 ts_tablet_manager.cc:1428] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:47:58.625756 14651 ts_tablet_manager.cc:1905] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:58.627895 14556 tablet_bootstrap.cc:492] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: No bootstrap required, opened a new log
I20260811 18:47:58.627962 14556 ts_tablet_manager.cc:1397] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260811 18:47:58.628140 14556 raft_consensus.cc:348] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.628239 14556 raft_consensus.cc:374] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:47:58.628266 14556 raft_consensus.cc:729] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee236172dc004dcdb84081776247f5de, State: Initialized, Role: FOLLOWER
I20260811 18:47:58.628311 14556 consensus_queue.cc:260] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.628417 14556 ts_tablet_manager.cc:1428] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 18:47:58.628991 14651 ts_tablet_manager.cc:1918] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:47:58.629118 14651 log.cc:1196] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/abea0ad929774957a23530e7e6fce015
I20260811 18:47:58.629434 14651 ts_tablet_manager.cc:1939] T abea0ad929774957a23530e7e6fce015 P 9f0ecd24231242ab8e296baa444cff3e: Deleting consensus metadata
I20260811 18:47:58.630085 13547 catalog_manager.cc:4841] TS 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307): tablet abea0ad929774957a23530e7e6fce015 (table test-table [id=b5d25edf6526497da0aa9dc42111288c]) successfully deleted
I20260811 18:47:58.631551 14652 tablet_replica.cc:330] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: stopping tablet replica
I20260811 18:47:58.631705 14652 raft_consensus.cc:2229] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:47:58.631778 14652 raft_consensus.cc:2258] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:47:58.632342 14652 ts_tablet_manager.cc:1905] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:47:58.634438 14652 ts_tablet_manager.cc:1918] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:47:58.634511 14652 log.cc:1196] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/abea0ad929774957a23530e7e6fce015
I20260811 18:47:58.634771 14652 ts_tablet_manager.cc:1939] T abea0ad929774957a23530e7e6fce015 P ee236172dc004dcdb84081776247f5de: Deleting consensus metadata
I20260811 18:47:58.635216 13541 catalog_manager.cc:4841] TS ee236172dc004dcdb84081776247f5de (127.9.215.2:37637): tablet abea0ad929774957a23530e7e6fce015 (table test-table [id=b5d25edf6526497da0aa9dc42111288c]) successfully deleted
I20260811 18:47:58.759799 14562 raft_consensus.cc:482] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:47:58.759912 14562 raft_consensus.cc:504] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.760365 14562 leader_election.cc:288] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533), ee236172dc004dcdb84081776247f5de (127.9.215.2:37637)
I20260811 18:47:58.763964 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.760796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:35955 (local address 127.9.215.1:38533)
0811 18:47:58.760865 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.760867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.760880 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:47:58.760912 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.760913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.760923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.760989 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.761346 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.761894 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.762996 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.763567 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.763641 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.763653 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.763844 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.763889 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:47:58.763971 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.760565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38533 (local address 127.9.215.3:35955)
0811 18:47:58.760743 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.760755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.760821 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.761008 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.761009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.761113 (+ 104us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.761267 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.761269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.761999 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.762000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.762909 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.762913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.763662 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.763663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.763741 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.763747 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.763831 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.763875 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260811 18:47:58.764079 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.760799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:57121 (local address 127.9.215.2:37637)
0811 18:47:58.760924 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.760926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.760934 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:47:58.760962 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.760963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.760972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.761021 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:47:58.761260 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.761918 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.763165 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.763730 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.763787 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.763799 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.763968 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.764020 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 18:47:58.764328 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.760571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:37637 (local address 127.9.215.3:57121)
0811 18:47:58.760840 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.760849 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.760894 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.761067 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.761068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.761072 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:47:58.761174 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.761176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.762017 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.762019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.763057 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.763061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.763805 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.763806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.763880 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.763885 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.764198 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.764249 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260811 18:47:58.764407 13886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5c1bec1bf9497fae4ad1a0cf0a12cd" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32623c3009747ed8ffcb9dc9a43342f" is_pre_election: true
I20260811 18:47:58.764525 13886 raft_consensus.cc:2454] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 0.
I20260811 18:47:58.764562 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5c1bec1bf9497fae4ad1a0cf0a12cd" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de" is_pre_election: true
I20260811 18:47:58.764647 14144 raft_consensus.cc:2454] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 0.
I20260811 18:47:58.764813 14291 leader_election.cc:302] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, b32623c3009747ed8ffcb9dc9a43342f; no voters:
I20260811 18:47:58.764950 14562 raft_consensus.cc:2781] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:47:58.765003 14562 raft_consensus.cc:482] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:47:58.765018 14562 raft_consensus.cc:3037] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.766225 14562 raft_consensus.cc:504] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.766364 14562 leader_election.cc:288] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 election: Requested vote from peers b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533), ee236172dc004dcdb84081776247f5de (127.9.215.2:37637)
I20260811 18:47:58.766562 13886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5c1bec1bf9497fae4ad1a0cf0a12cd" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32623c3009747ed8ffcb9dc9a43342f"
I20260811 18:47:58.766575 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5c1bec1bf9497fae4ad1a0cf0a12cd" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de"
I20260811 18:47:58.766649 14144 raft_consensus.cc:3037] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.766647 13886 raft_consensus.cc:3037] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:47:58.767258 14144 raft_consensus.cc:2454] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 1.
I20260811 18:47:58.767441 13886 raft_consensus.cc:2454] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 1.
I20260811 18:47:58.767442 14284 leader_election.cc:302] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, ee236172dc004dcdb84081776247f5de; no voters:
I20260811 18:47:58.767552 14562 raft_consensus.cc:2781] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:47:58.767668 14562 raft_consensus.cc:686] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 1 LEADER]: Becoming Leader. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Running, Role: LEADER
I20260811 18:47:58.767743 14562 consensus_queue.cc:237] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:47:58.768473 13577 catalog_manager.cc:5495] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3). New cstate: current_term: 1 leader_uuid: "9f0ecd24231242ab8e296baa444cff3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } health_report { overall_health: HEALTHY } } }
I20260811 18:47:58.775213 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.773201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.9.215.3:34307)
0811 18:47:58.773266 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.773268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.773275 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:47:58.773309 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.773309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.773327 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.773389 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.773469 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.773944 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.774619 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.774796 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.774839 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.774936 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.775040 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.775088 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.775108 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.775126 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:47:58.780478 14549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.778261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:43235 (local address 127.9.215.2:37637)
0811 18:47:58.778344 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.778347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.778455 (+ 108us) server_negotiation.cc:407] Connection header received
0811 18:47:58.778498 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.778499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.778514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.778558 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.778659 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.779130 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.779917 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.780119 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.780174 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.780301 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.780327 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.780359 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.780397 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.780416 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:47:58.780508 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.778184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:37637 (local address 127.9.215.3:43235)
0811 18:47:58.778391 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.778401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.778457 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.778559 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.778559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.778561 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:58.778624 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.778625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.779188 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.779189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.779830 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.779833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.780181 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.780181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.780228 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.780243 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.780365 (+ 122us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.780366 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.780394 (+ 28us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.780429 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":134,"threads_started":1}
I20260811 18:47:58.782876 13886 raft_consensus.cc:1261] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Refusing update from remote peer 9f0ecd24231242ab8e296baa444cff3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:58.782903 14144 raft_consensus.cc:1261] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Refusing update from remote peer 9f0ecd24231242ab8e296baa444cff3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:47:58.783267 14665 consensus_queue.cc:1035] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:58.783407 14562 consensus_queue.cc:1035] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:47:58.791311 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.788525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:34307 (local address 127.9.215.2:47285)
0811 18:47:58.788594 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 18:47:58.788606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:47:58.788643 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:47:58.788814 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:47:58.788814 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:47:58.788815 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:47:58.788867 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.788869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.789652 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.789653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.790527 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:47:58.790531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.790913 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.790914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:47:58.790973 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.790979 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.791136 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.791138 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:47:58.791178 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 18:47:58.791232 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 18:47:58.791528 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:47:58.788671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:47285 (local address 127.9.215.3:34307)
0811 18:47:58.788711 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:47:58.788713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:47:58.788720 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:47:58.788743 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:47:58.788743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:47:58.788753 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:47:58.788803 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:47:58.788947 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.789592 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.790603 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:47:58.790848 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:47:58.790903 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:47:58.791036 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:47:58.791089 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:47:58.791129 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 18:47:58.791409 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:47:58.791432 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 18:47:58.791872 14610 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:48:00.313117 13571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:00.313228 13571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:00.314110 13571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Sending DeleteTablet for 3 replicas of tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd
I20260811 18:48:00.314393 14379 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:00 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:53498
I20260811 18:48:00.314399 13844 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:00 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:47674
I20260811 18:48:00.314625 14730 tablet_replica.cc:330] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: stopping tablet replica
I20260811 18:48:00.314711 14730 raft_consensus.cc:2229] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:00.314689 14107 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:00 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36524
I20260811 18:48:00.314759 14730 raft_consensus.cc:2258] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:00.314860 14728 tablet_replica.cc:330] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: stopping tablet replica
I20260811 18:48:00.314913 14728 raft_consensus.cc:2229] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:00.314931 14732 tablet_replica.cc:330] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: stopping tablet replica
I20260811 18:48:00.315059 14728 raft_consensus.cc:2258] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:00.315070 14732 raft_consensus.cc:2229] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:00.315168 14732 raft_consensus.cc:2258] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:00.315490 14732 ts_tablet_manager.cc:1905] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:00.317253 14732 ts_tablet_manager.cc:1918] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:48:00.317327 14732 log.cc:1196] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/8c5c1bec1bf9497fae4ad1a0cf0a12cd
I20260811 18:48:00.317521 14732 ts_tablet_manager.cc:1939] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P ee236172dc004dcdb84081776247f5de: Deleting consensus metadata
I20260811 18:48:00.317994 13541 catalog_manager.cc:4841] TS ee236172dc004dcdb84081776247f5de (127.9.215.2:37637): tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd (table test-table [id=ea63baa04101436a87df180e042f109b]) successfully deleted
I20260811 18:48:00.318141 14728 ts_tablet_manager.cc:1905] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:00.319610 14728 ts_tablet_manager.cc:1918] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:48:00.319689 14728 log.cc:1196] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/8c5c1bec1bf9497fae4ad1a0cf0a12cd
I20260811 18:48:00.319702 14730 ts_tablet_manager.cc:1905] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:00.319958 14728 ts_tablet_manager.cc:1939] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P 9f0ecd24231242ab8e296baa444cff3e: Deleting consensus metadata
I20260811 18:48:00.320377 13547 catalog_manager.cc:4841] TS 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307): tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd (table test-table [id=ea63baa04101436a87df180e042f109b]) successfully deleted
I20260811 18:48:00.321255 14730 ts_tablet_manager.cc:1918] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:48:00.321317 14730 log.cc:1196] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/8c5c1bec1bf9497fae4ad1a0cf0a12cd
I20260811 18:48:00.321518 14730 ts_tablet_manager.cc:1939] T 8c5c1bec1bf9497fae4ad1a0cf0a12cd P b32623c3009747ed8ffcb9dc9a43342f: Deleting consensus metadata
I20260811 18:48:00.321898 13547 catalog_manager.cc:4841] TS b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533): tablet 8c5c1bec1bf9497fae4ad1a0cf0a12cd (table test-table [id=ea63baa04101436a87df180e042f109b]) successfully deleted
Aug 11 18:48:00 dist-test-slave-pww0 krb5kdc[13497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:48:00 dist-test-slave-pww0 krb5kdc[13497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:00.342067 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:00.334569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.9.215.62:43649)
0811 18:48:00.334708 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:00.334710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:00.334830 (+ 120us) server_negotiation.cc:407] Connection header received
0811 18:48:00.334902 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:00.334903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:00.334913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:00.334988 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:00.335800 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:00.336340 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:00.337101 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:00.337320 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:00.337389 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:00.339645 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:00.339647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:00.339647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:00.339656 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:00.341136 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:00.341484 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:00.341485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:00.341485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:00.341540 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:00.341667 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:00.341668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:00.341668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:00.341775 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:00.341849 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:00.341922 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 18:48:00.341990 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260811 18:48:00.343730 13571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:00.343897 13571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:00.345293 13844 tablet_service.cc:1467] Processing CreateTablet for tablet 0c008429499e4c47bca52155775a0bc4 (DEFAULT_TABLE table=test-table [id=88abc24f51c54f708ce4d4d5b364a6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:00.345320 14107 tablet_service.cc:1467] Processing CreateTablet for tablet 0c008429499e4c47bca52155775a0bc4 (DEFAULT_TABLE table=test-table [id=88abc24f51c54f708ce4d4d5b364a6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:00.345449 14107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c008429499e4c47bca52155775a0bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:00.345453 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c008429499e4c47bca52155775a0bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:00.346915 14750 tablet_bootstrap.cc:492] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Bootstrap starting.
I20260811 18:48:00.346915 14749 tablet_bootstrap.cc:492] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Bootstrap starting.
I20260811 18:48:00.346944 14379 tablet_service.cc:1467] Processing CreateTablet for tablet 0c008429499e4c47bca52155775a0bc4 (DEFAULT_TABLE table=test-table [id=88abc24f51c54f708ce4d4d5b364a6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:00.347074 14379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c008429499e4c47bca52155775a0bc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:00.347378 14749 tablet_bootstrap.cc:654] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:00.347549 14750 tablet_bootstrap.cc:654] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:00.348217 14749 tablet_bootstrap.cc:492] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: No bootstrap required, opened a new log
I20260811 18:48:00.348281 14749 ts_tablet_manager.cc:1397] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:00.348436 14750 tablet_bootstrap.cc:492] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: No bootstrap required, opened a new log
I20260811 18:48:00.348487 14750 ts_tablet_manager.cc:1397] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:48:00.348443 14749 raft_consensus.cc:348] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.348541 14749 raft_consensus.cc:374] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:00.348563 14749 raft_consensus.cc:729] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Initialized, Role: FOLLOWER
I20260811 18:48:00.348588 14756 tablet_bootstrap.cc:492] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Bootstrap starting.
I20260811 18:48:00.348606 14749 consensus_queue.cc:260] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.348623 14750 raft_consensus.cc:348] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.348693 14750 raft_consensus.cc:374] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:00.348726 14750 raft_consensus.cc:729] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee236172dc004dcdb84081776247f5de, State: Initialized, Role: FOLLOWER
I20260811 18:48:00.348785 14749 ts_tablet_manager.cc:1428] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:00.348788 14750 consensus_queue.cc:260] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.348881 14750 ts_tablet_manager.cc:1428] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:00.349150 14756 tablet_bootstrap.cc:654] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:00.349828 14756 tablet_bootstrap.cc:492] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: No bootstrap required, opened a new log
I20260811 18:48:00.349877 14756 ts_tablet_manager.cc:1397] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:00.350019 14756 raft_consensus.cc:348] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.350078 14756 raft_consensus.cc:374] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:00.350096 14756 raft_consensus.cc:729] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Initialized, Role: FOLLOWER
I20260811 18:48:00.350148 14756 consensus_queue.cc:260] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.350246 14756 ts_tablet_manager.cc:1428] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:00.487627 14681 raft_consensus.cc:482] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:00.487747 14681 raft_consensus.cc:504] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.487928 14681 leader_election.cc:288] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533), ee236172dc004dcdb84081776247f5de (127.9.215.2:37637)
I20260811 18:48:00.488190 13886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c008429499e4c47bca52155775a0bc4" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32623c3009747ed8ffcb9dc9a43342f" is_pre_election: true
I20260811 18:48:00.488209 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c008429499e4c47bca52155775a0bc4" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de" is_pre_election: true
I20260811 18:48:00.488296 13886 raft_consensus.cc:2454] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 0.
I20260811 18:48:00.488314 14144 raft_consensus.cc:2454] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 0.
I20260811 18:48:00.488488 14291 leader_election.cc:302] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, b32623c3009747ed8ffcb9dc9a43342f; no voters:
I20260811 18:48:00.488598 14681 raft_consensus.cc:2781] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:00.488662 14681 raft_consensus.cc:482] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:00.488683 14681 raft_consensus.cc:3037] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:00.489423 14681 raft_consensus.cc:504] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.489557 14681 leader_election.cc:288] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 election: Requested vote from peers b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533), ee236172dc004dcdb84081776247f5de (127.9.215.2:37637)
I20260811 18:48:00.489804 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c008429499e4c47bca52155775a0bc4" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de"
I20260811 18:48:00.489825 13886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c008429499e4c47bca52155775a0bc4" candidate_uuid: "9f0ecd24231242ab8e296baa444cff3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32623c3009747ed8ffcb9dc9a43342f"
I20260811 18:48:00.489886 14144 raft_consensus.cc:3037] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:00.489913 13886 raft_consensus.cc:3037] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:00.490662 13886 raft_consensus.cc:2454] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 1.
I20260811 18:48:00.490658 14144 raft_consensus.cc:2454] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f0ecd24231242ab8e296baa444cff3e in term 1.
I20260811 18:48:00.490845 14291 leader_election.cc:302] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, b32623c3009747ed8ffcb9dc9a43342f; no voters:
I20260811 18:48:00.490953 14681 raft_consensus.cc:2781] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:00.491019 14681 raft_consensus.cc:686] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 1 LEADER]: Becoming Leader. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Running, Role: LEADER
I20260811 18:48:00.491074 14681 consensus_queue.cc:237] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.491753 13573 catalog_manager.cc:5495] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3). New cstate: current_term: 1 leader_uuid: "9f0ecd24231242ab8e296baa444cff3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } health_report { overall_health: HEALTHY } } }
I20260811 18:48:00.498605 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:00.496277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.9.215.3:34307)
0811 18:48:00.496425 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:00.496427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:00.496473 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:48:00.496548 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:00.496548 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:00.496560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:00.496616 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:00.496721 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:00.497215 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:00.497956 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:00.498170 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:00.498224 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:00.498356 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:00.498395 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:00.498439 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:00.498495 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 18:48:00.498518 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":97,"threads_started":1}
I20260811 18:48:00.500964 13886 raft_consensus.cc:1261] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Refusing update from remote peer 9f0ecd24231242ab8e296baa444cff3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:00.501188 14144 raft_consensus.cc:1261] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Refusing update from remote peer 9f0ecd24231242ab8e296baa444cff3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:00.501209 14715 consensus_queue.cc:1035] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:00.501389 14715 consensus_queue.cc:1035] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:00.511412 13573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:00.511560 13573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:00.512153 13573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Sending DeleteTablet for 3 replicas of tablet 0c008429499e4c47bca52155775a0bc4
I20260811 18:48:00.512427 13844 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c008429499e4c47bca52155775a0bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:00 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:47674
I20260811 18:48:00.512455 14107 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c008429499e4c47bca52155775a0bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:00 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36524
I20260811 18:48:00.512452 14379 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c008429499e4c47bca52155775a0bc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:00 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:53498
I20260811 18:48:00.512552 14730 tablet_replica.cc:330] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: stopping tablet replica
I20260811 18:48:00.512573 14732 tablet_replica.cc:330] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: stopping tablet replica
I20260811 18:48:00.512630 14730 raft_consensus.cc:2229] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:00.512643 14732 raft_consensus.cc:2229] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:00.512683 14732 raft_consensus.cc:2258] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:00.512683 14730 raft_consensus.cc:2258] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:00.512784 14728 tablet_replica.cc:330] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: stopping tablet replica
I20260811 18:48:00.512847 14728 raft_consensus.cc:2229] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:00.512914 14728 raft_consensus.cc:2258] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:00.513057 14732 ts_tablet_manager.cc:1905] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:00.513111 14730 ts_tablet_manager.cc:1905] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:00.513178 14728 ts_tablet_manager.cc:1905] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:00.514392 13573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:00.514555 13573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:00.514621 14728 ts_tablet_manager.cc:1918] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:00.514689 14728 log.cc:1196] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/0c008429499e4c47bca52155775a0bc4
I20260811 18:48:00.514729 14730 ts_tablet_manager.cc:1918] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:00.514776 14730 log.cc:1196] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/0c008429499e4c47bca52155775a0bc4
I20260811 18:48:00.514793 14732 ts_tablet_manager.cc:1918] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:00.514846 14732 log.cc:1196] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/0c008429499e4c47bca52155775a0bc4
I20260811 18:48:00.514915 14728 ts_tablet_manager.cc:1939] T 0c008429499e4c47bca52155775a0bc4 P 9f0ecd24231242ab8e296baa444cff3e: Deleting consensus metadata
I20260811 18:48:00.514986 14730 ts_tablet_manager.cc:1939] T 0c008429499e4c47bca52155775a0bc4 P b32623c3009747ed8ffcb9dc9a43342f: Deleting consensus metadata
I20260811 18:48:00.515069 14732 ts_tablet_manager.cc:1939] T 0c008429499e4c47bca52155775a0bc4 P ee236172dc004dcdb84081776247f5de: Deleting consensus metadata
I20260811 18:48:00.515286 13547 catalog_manager.cc:4841] TS b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533): tablet 0c008429499e4c47bca52155775a0bc4 (table test-table [id=88abc24f51c54f708ce4d4d5b364a6c8]) successfully deleted
I20260811 18:48:00.515367 13547 catalog_manager.cc:4841] TS 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307): tablet 0c008429499e4c47bca52155775a0bc4 (table test-table [id=88abc24f51c54f708ce4d4d5b364a6c8]) successfully deleted
I20260811 18:48:00.515393 13541 catalog_manager.cc:4841] TS ee236172dc004dcdb84081776247f5de (127.9.215.2:37637): tablet 0c008429499e4c47bca52155775a0bc4 (table test-table [id=88abc24f51c54f708ce4d4d5b364a6c8]) successfully deleted
I20260811 18:48:00.516101 14107 tablet_service.cc:1467] Processing CreateTablet for tablet d1e390b92cf144669f553f9502f6847a (DEFAULT_TABLE table=test-table [id=f606b8950860478f95f561564d15d04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:00.516098 14379 tablet_service.cc:1467] Processing CreateTablet for tablet d1e390b92cf144669f553f9502f6847a (DEFAULT_TABLE table=test-table [id=f606b8950860478f95f561564d15d04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:00.516098 13844 tablet_service.cc:1467] Processing CreateTablet for tablet d1e390b92cf144669f553f9502f6847a (DEFAULT_TABLE table=test-table [id=f606b8950860478f95f561564d15d04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:00.516211 13844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e390b92cf144669f553f9502f6847a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:00.516211 14379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e390b92cf144669f553f9502f6847a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:00.516217 14107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e390b92cf144669f553f9502f6847a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:00.517102 14750 tablet_bootstrap.cc:492] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Bootstrap starting.
I20260811 18:48:00.517373 14756 tablet_bootstrap.cc:492] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Bootstrap starting.
I20260811 18:48:00.517467 14750 tablet_bootstrap.cc:654] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:00.517692 14756 tablet_bootstrap.cc:654] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:00.517970 14749 tablet_bootstrap.cc:492] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f: Bootstrap starting.
I20260811 18:48:00.518066 14750 tablet_bootstrap.cc:492] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: No bootstrap required, opened a new log
I20260811 18:48:00.518115 14750 ts_tablet_manager.cc:1397] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:00.518210 14750 raft_consensus.cc:348] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.518261 14750 raft_consensus.cc:374] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:00.518263 14749 tablet_bootstrap.cc:654] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:00.518273 14750 raft_consensus.cc:729] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee236172dc004dcdb84081776247f5de, State: Initialized, Role: FOLLOWER
I20260811 18:48:00.518308 14750 consensus_queue.cc:260] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.518369 14750 ts_tablet_manager.cc:1428] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:00.518736 14756 tablet_bootstrap.cc:492] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: No bootstrap required, opened a new log
I20260811 18:48:00.518790 14756 ts_tablet_manager.cc:1397] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:00.518929 14756 raft_consensus.cc:348] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.519004 14756 raft_consensus.cc:374] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:00.519024 14756 raft_consensus.cc:729] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0ecd24231242ab8e296baa444cff3e, State: Initialized, Role: FOLLOWER
I20260811 18:48:00.519071 14756 consensus_queue.cc:260] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.519160 14756 ts_tablet_manager.cc:1428] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:00.519510 14749 tablet_bootstrap.cc:492] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f: No bootstrap required, opened a new log
I20260811 18:48:00.519562 14749 ts_tablet_manager.cc:1397] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:48:00.519665 14749 raft_consensus.cc:348] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.519737 14749 raft_consensus.cc:374] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:00.519757 14749 raft_consensus.cc:729] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Initialized, Role: FOLLOWER
I20260811 18:48:00.519804 14749 consensus_queue.cc:260] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.519898 14749 ts_tablet_manager.cc:1428] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:00.747694 14757 raft_consensus.cc:482] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:00.747808 14757 raft_consensus.cc:504] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.747987 14757 leader_election.cc:288] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee236172dc004dcdb84081776247f5de (127.9.215.2:37637), 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:48:00.748272 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e390b92cf144669f553f9502f6847a" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0ecd24231242ab8e296baa444cff3e" is_pre_election: true
I20260811 18:48:00.748274 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e390b92cf144669f553f9502f6847a" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de" is_pre_election: true
I20260811 18:48:00.748366 14422 raft_consensus.cc:2454] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 0.
I20260811 18:48:00.748383 14144 raft_consensus.cc:2454] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 0.
I20260811 18:48:00.748534 13748 leader_election.cc:302] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, b32623c3009747ed8ffcb9dc9a43342f; no voters:
I20260811 18:48:00.748668 14757 raft_consensus.cc:2781] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:00.748713 14757 raft_consensus.cc:482] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:00.748728 14757 raft_consensus.cc:3037] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:00.749425 14757 raft_consensus.cc:504] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.749560 14757 leader_election.cc:288] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 election: Requested vote from peers ee236172dc004dcdb84081776247f5de (127.9.215.2:37637), 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307)
I20260811 18:48:00.749754 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e390b92cf144669f553f9502f6847a" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0ecd24231242ab8e296baa444cff3e"
I20260811 18:48:00.749754 14144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e390b92cf144669f553f9502f6847a" candidate_uuid: "b32623c3009747ed8ffcb9dc9a43342f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee236172dc004dcdb84081776247f5de"
I20260811 18:48:00.749855 14422 raft_consensus.cc:3037] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:00.749855 14144 raft_consensus.cc:3037] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:00.750660 14422 raft_consensus.cc:2454] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 1.
I20260811 18:48:00.750660 14144 raft_consensus.cc:2454] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32623c3009747ed8ffcb9dc9a43342f in term 1.
I20260811 18:48:00.750913 13748 leader_election.cc:302] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f0ecd24231242ab8e296baa444cff3e, b32623c3009747ed8ffcb9dc9a43342f; no voters:
I20260811 18:48:00.751039 14757 raft_consensus.cc:2781] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:00.751237 14757 raft_consensus.cc:686] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [term 1 LEADER]: Becoming Leader. State: Replica: b32623c3009747ed8ffcb9dc9a43342f, State: Running, Role: LEADER
I20260811 18:48:00.751298 14757 consensus_queue.cc:237] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } }
I20260811 18:48:00.751969 13573 catalog_manager.cc:5495] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f reported cstate change: term changed from 0 to 1, leader changed from <none> to b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1). New cstate: current_term: 1 leader_uuid: "b32623c3009747ed8ffcb9dc9a43342f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32623c3009747ed8ffcb9dc9a43342f" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 } health_report { overall_health: UNKNOWN } } }
I20260811 18:48:00.807484 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:00.805172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.9.215.1:38533)
0811 18:48:00.805365 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:00.805367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:00.805373 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:48:00.805391 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:00.805392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:00.805402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:00.805460 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:00.805589 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:00.806080 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:00.806810 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:00.807017 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:00.807073 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:00.807207 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:00.807246 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:00.807287 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:00.807342 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 18:48:00.807365 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
I20260811 18:48:00.810009 14422 raft_consensus.cc:1261] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Refusing update from remote peer b32623c3009747ed8ffcb9dc9a43342f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:00.810042 14144 raft_consensus.cc:1261] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Refusing update from remote peer b32623c3009747ed8ffcb9dc9a43342f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:00.810252 14779 consensus_queue.cc:1035] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0ecd24231242ab8e296baa444cff3e" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 34307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:00.810403 14757 consensus_queue.cc:1035] T d1e390b92cf144669f553f9502f6847a P b32623c3009747ed8ffcb9dc9a43342f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee236172dc004dcdb84081776247f5de" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 37637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:48:00.815029 14687 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:48:02.332791 13571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:02.332912 13571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:02.333971 13571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d7d0f300396497ab4bec6636ee1d4e0: Sending DeleteTablet for 3 replicas of tablet d1e390b92cf144669f553f9502f6847a
I20260811 18:48:02.334263 13844 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e390b92cf144669f553f9502f6847a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:02 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:47674
I20260811 18:48:02.334298 14379 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e390b92cf144669f553f9502f6847a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:02 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:53498
I20260811 18:48:02.334399 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13717
I20260811 18:48:02.334473 14107 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e390b92cf144669f553f9502f6847a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:02 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36524
I20260811 18:48:02.344061 14830 tablet_replica.cc:330] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: stopping tablet replica
I20260811 18:48:02.344173 14830 raft_consensus.cc:2229] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:02.344229 14830 raft_consensus.cc:2258] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:02.347000 14830 ts_tablet_manager.cc:1905] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:02.349148 14830 ts_tablet_manager.cc:1918] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:48:02.349241 14830 log.cc:1196] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/d1e390b92cf144669f553f9502f6847a
I20260811 18:48:02.349479 14830 ts_tablet_manager.cc:1939] T d1e390b92cf144669f553f9502f6847a P 9f0ecd24231242ab8e296baa444cff3e: Deleting consensus metadata
I20260811 18:48:02.349638 14833 tablet_replica.cc:330] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: stopping tablet replica
I20260811 18:48:02.349704 14833 raft_consensus.cc:2229] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:02.349748 14833 raft_consensus.cc:2258] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:02.349941 13547 catalog_manager.cc:4841] TS 9f0ecd24231242ab8e296baa444cff3e (127.9.215.3:34307): tablet d1e390b92cf144669f553f9502f6847a (table test-table [id=f606b8950860478f95f561564d15d04d]) successfully deleted
I20260811 18:48:02.352145 14833 ts_tablet_manager.cc:1905] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:02.353549 14833 ts_tablet_manager.cc:1918] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:48:02.353613 14833 log.cc:1196] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/d1e390b92cf144669f553f9502f6847a
I20260811 18:48:02.353806 14833 ts_tablet_manager.cc:1939] T d1e390b92cf144669f553f9502f6847a P ee236172dc004dcdb84081776247f5de: Deleting consensus metadata
I20260811 18:48:02.354450 13541 catalog_manager.cc:4841] TS ee236172dc004dcdb84081776247f5de (127.9.215.2:37637): tablet d1e390b92cf144669f553f9502f6847a (table test-table [id=f606b8950860478f95f561564d15d04d]) successfully deleted
W20260811 18:48:02.357465 13547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:38533 (error 108)
I20260811 18:48:02.357676 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13975
I20260811 18:48:02.368009 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:02.367684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38533 (local address 127.0.0.1:48950)
0811 18:48:02.367887 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:02.367960 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:38533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
W20260811 18:48:02.368175 13547 catalog_manager.cc:4584] TS b32623c3009747ed8ffcb9dc9a43342f (127.9.215.1:38533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1e390b92cf144669f553f9502f6847a: Network error: Client connection negotiation failed: client connection to 127.9.215.1:38533: connect: Connection refused (error 111)
I20260811 18:48:02.372771 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 14250
I20260811 18:48:02.390028 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13514
2026-08-11T18:48:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6291 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:02 dist-test-slave-pww0 krb5kdc[14841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:02 dist-test-slave-pww0 krb5kdc[14841](info): set up 2 sockets
Aug 11 18:48:02 dist-test-slave-pww0 krb5kdc[14841](info): commencing operation
krb5kdc: starting...
W20260811 18:48:03.635932 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:03 dist-test-slave-pww0 krb5kdc[14841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:03.762801 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45281
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:45281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:03.846680 14868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:03.846923 14868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:03.846958 14868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:03.848234 14868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:03.848289 14868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:03.848308 14868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:03.848325 14868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:03.848340 14868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:03.849934 14868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:45281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:03.850214 14868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:03.850463 14868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:03.856220 14883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:03.856207 14880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:03.856277 14881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:03.856244 14868 server_base.cc:1034] running on GCE node
I20260811 18:48:03.856669 14868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:03.856980 14868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:03.858155 14868 hybrid_clock.cc:648] HybridClock initialized: now 1786474083858133 us; error 36 us; skew 500 ppm
Aug 11 18:48:03 dist-test-slave-pww0 krb5kdc[14841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474083, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:03.866236 14868 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:03.866819 14868 webserver.cc:460] Webserver started at http://127.9.215.62:39667/ using document root <none> and password file <none>
I20260811 18:48:03.867049 14868 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:03.867101 14868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:03.867236 14868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:03.868429 14868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "57e171755500440b8aa773ad676b2aa9"
format_stamp: "Formatted at 2026-08-11 18:48:03 on dist-test-slave-pww0"
server_key: "573f33924caaefeba22d2d82be900706"
server_key_iv: "788f25823cbadbcf57485eac2a4a4250"
server_key_version: "encryptionkey@0"
I20260811 18:48:03.868808 14868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "57e171755500440b8aa773ad676b2aa9"
format_stamp: "Formatted at 2026-08-11 18:48:03 on dist-test-slave-pww0"
server_key: "573f33924caaefeba22d2d82be900706"
server_key_iv: "788f25823cbadbcf57485eac2a4a4250"
server_key_version: "encryptionkey@0"
I20260811 18:48:03.870371 14868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:03.871574 14897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:03.871793 14868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:03.871905 14868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "57e171755500440b8aa773ad676b2aa9"
format_stamp: "Formatted at 2026-08-11 18:48:03 on dist-test-slave-pww0"
server_key: "573f33924caaefeba22d2d82be900706"
server_key_iv: "788f25823cbadbcf57485eac2a4a4250"
server_key_version: "encryptionkey@0"
I20260811 18:48:03.871989 14868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:03.883448 14868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:03.885838 14868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:03.885989 14868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:03.890583 14868 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:45281
I20260811 18:48:03.890625 14997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:45281 every 8 connection(s)
I20260811 18:48:03.891008 14868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:03.891947 14998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:03.894397 14998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9: Bootstrap starting.
I20260811 18:48:03.894979 14998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:03.895229 14998 log.cc:824] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:03.895987 14998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9: No bootstrap required, opened a new log
I20260811 18:48:03.897668 14998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e171755500440b8aa773ad676b2aa9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45281 } }
I20260811 18:48:03.897826 14998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:03.897850 14998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e171755500440b8aa773ad676b2aa9, State: Initialized, Role: FOLLOWER
I20260811 18:48:03.897976 14998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e171755500440b8aa773ad676b2aa9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45281 } }
I20260811 18:48:03.898057 14998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:03.898087 14998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:03.898123 14998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:03.898699 14998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e171755500440b8aa773ad676b2aa9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45281 } }
I20260811 18:48:03.898814 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 14868
I20260811 18:48:03.898825 14998 leader_election.cc:302] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57e171755500440b8aa773ad676b2aa9; no voters:
I20260811 18:48:03.898905 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:03.899021 14998 leader_election.cc:288] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:03.899070 15004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:03.899101 10076 external_mini_cluster.cc:1428] Setting key 7d1519b86680c5c1880707a894ba2d2c
I20260811 18:48:03.899253 15004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [term 1 LEADER]: Becoming Leader. State: Replica: 57e171755500440b8aa773ad676b2aa9, State: Running, Role: LEADER
I20260811 18:48:03.899333 14998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:03.899418 15004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e171755500440b8aa773ad676b2aa9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45281 } }
I20260811 18:48:03.899823 15009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57e171755500440b8aa773ad676b2aa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e171755500440b8aa773ad676b2aa9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45281 } } }
I20260811 18:48:03.899912 15009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:03.900049 15007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57e171755500440b8aa773ad676b2aa9. Latest consensus state: current_term: 1 leader_uuid: "57e171755500440b8aa773ad676b2aa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57e171755500440b8aa773ad676b2aa9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45281 } } }
I20260811 18:48:03.900141 15016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:03.900144 15007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:03.901005 15016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:03.902844 15016 catalog_manager.cc:1347] Generated new cluster ID: 1a9a97ee509b4e30aedab862ba97bfc1
I20260811 18:48:03.902911 15016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:48:03 dist-test-slave-pww0 krb5kdc[14841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:03.912518 15016 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:03.912554 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:03.899841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.9.215.62:45281)
0811 18:48:03.900090 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:03.900101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:03.900228 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:48:03.900378 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:03.900387 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:03.900453 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:03.901650 (+ 1197us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:03.901770 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:03.902982 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:03.904138 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:03.904798 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:03.905238 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:03.907727 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:03.907733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:03.907733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:03.907863 (+ 130us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:03.910109 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:03.910627 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:03.910631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:03.910631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:03.910781 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:03.911027 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:03.911028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:03.911028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:03.911365 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:03.912039 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:03.912108 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 18:48:03.912275 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":72,"threads_started":1}
I20260811 18:48:03.913182 15016 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:03.917514 15016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57e171755500440b8aa773ad676b2aa9: Generated new TSK 0
I20260811 18:48:03.917729 15016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:03.948643 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45281
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:04.036325 15043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:04.036559 15043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:04.036586 15043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:04.037830 15043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:04.037884 15043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:04.037914 15043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:04.039517 15043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:04.039784 15043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:04.040036 15043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:04.045531 15056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:04.045560 15059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:04.045531 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:04.045853 15043 server_base.cc:1034] running on GCE node
I20260811 18:48:04.046078 15043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:04.046459 15043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:04.047631 15043 hybrid_clock.cc:648] HybridClock initialized: now 1786474084047605 us; error 35 us; skew 500 ppm
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[14841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:04.050623 15043 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:04.051193 15043 webserver.cc:460] Webserver started at http://127.9.215.1:36525/ using document root <none> and password file <none>
I20260811 18:48:04.051447 15043 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:04.051548 15043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:04.051752 15043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:04.052901 15043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "44bc0cc32f5d4b08bc08b445ebbaf40c"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "2aeede122863f0122758fce3d7f8c84e"
server_key_iv: "6335429044b710844d7cc49060541ea0"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.053362 15043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "44bc0cc32f5d4b08bc08b445ebbaf40c"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "2aeede122863f0122758fce3d7f8c84e"
server_key_iv: "6335429044b710844d7cc49060541ea0"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.054848 15043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:04.055719 15080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.055876 15043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:04.055977 15043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "44bc0cc32f5d4b08bc08b445ebbaf40c"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "2aeede122863f0122758fce3d7f8c84e"
server_key_iv: "6335429044b710844d7cc49060541ea0"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.056044 15043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:04.084019 15043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:04.086102 15043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:04.086251 15043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:04.086498 15043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:04.086827 15043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:04.086862 15043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.086894 15043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:04.086917 15043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.092763 15043 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:37593
I20260811 18:48:04.092816 15315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:37593 every 8 connection(s)
I20260811 18:48:04.093161 15043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:04.095198 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15043
I20260811 18:48:04.095278 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:04.095446 10076 external_mini_cluster.cc:1428] Setting key 00c4f4380249da380d72d6c9fdd2e264
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[14841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:04.104555 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:04.093960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:56451 (local address 127.9.215.62:45281)
0811 18:48:04.094023 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:04.094026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:04.094246 (+ 220us) server_negotiation.cc:407] Connection header received
0811 18:48:04.094773 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:04.094774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:04.094788 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:04.094840 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:04.095973 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.096648 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.097722 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.097994 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.098055 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:04.101006 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:04.101008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:04.101009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:04.101017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:04.102768 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:04.103315 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:04.103318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:04.103318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:04.103383 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:04.103727 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:04.103728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:04.103728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:04.103879 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:04.103923 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:04.104338 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 18:48:04.104447 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:48:04.105149 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:04.093948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45281 (local address 127.9.215.1:56451)
0811 18:48:04.094124 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:04.094163 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:04.094683 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:04.094898 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:04.094901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:04.095594 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:04.095882 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:04.095891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.096777 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.096778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:04.097620 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:04.097623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.098082 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.098083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:04.098247 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:04.098850 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:04.098862 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:04.100827 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:04.102876 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:04.102887 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:04.102890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:04.103221 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:04.103481 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:04.103482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:04.103482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:04.103644 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:04.103974 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:04.103982 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:04.104169 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 18:48:04.104685 (+ 516us) client_negotiation.cc:240] Negotiation successful
0811 18:48:04.104892 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:48:04.105883 15316 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45281
I20260811 18:48:04.105997 15316 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:04.106209 15316 heartbeater.cc:511] Master 127.9.215.62:45281 requested a full tablet report, sending...
I20260811 18:48:04.106829 14931 ts_manager.cc:194] Registered new tserver with Master: 44bc0cc32f5d4b08bc08b445ebbaf40c (127.9.215.1:37593)
I20260811 18:48:04.107532 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:56451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:04.124543 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45281
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:04.208814 15322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:04.209048 15322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:04.209072 15322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:04.210289 15322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:04.210328 15322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:04.210354 15322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:04.211923 15322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:04.212165 15322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:04.212370 15322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:04.217691 15340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:04.217677 15344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:04.217742 15322 server_base.cc:1034] running on GCE node
W20260811 18:48:04.217677 15346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:04.218112 15322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:04.218372 15322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:04.219519 15322 hybrid_clock.cc:648] HybridClock initialized: now 1786474084219505 us; error 28 us; skew 500 ppm
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[14841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:04.222083 15322 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:04.222543 15322 webserver.cc:460] Webserver started at http://127.9.215.2:45563/ using document root <none> and password file <none>
I20260811 18:48:04.222764 15322 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:04.222807 15322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:04.222905 15322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:04.224020 15322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "9197f8f2291c4192a9c8b98038e38e8a"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "3305e2861d617010dc904e928fd82d10"
server_key_iv: "61a221a4ad5b3d2a1ea9f400c518c1f5"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.224344 15322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "9197f8f2291c4192a9c8b98038e38e8a"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "3305e2861d617010dc904e928fd82d10"
server_key_iv: "61a221a4ad5b3d2a1ea9f400c518c1f5"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.225711 15322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:48:04.226475 15355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.226673 15322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:04.226773 15322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "9197f8f2291c4192a9c8b98038e38e8a"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "3305e2861d617010dc904e928fd82d10"
server_key_iv: "61a221a4ad5b3d2a1ea9f400c518c1f5"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.226855 15322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:04.267048 15322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:04.269057 15322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:04.269194 15322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:04.269459 15322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:04.269794 15322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:04.269831 15322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.269856 15322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:04.269881 15322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.275894 15322 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:36301
I20260811 18:48:04.275997 15586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:36301 every 8 connection(s)
I20260811 18:48:04.276292 15322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:04.280874 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15322
I20260811 18:48:04.280972 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:04.281152 10076 external_mini_cluster.cc:1428] Setting key 192fc8ac374b5a3af6ba64b8a5f2073a
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[14841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:04.285805 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:04.277042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:54061 (local address 127.9.215.62:45281)
0811 18:48:04.277107 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:04.277109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:04.277388 (+ 279us) server_negotiation.cc:407] Connection header received
0811 18:48:04.277935 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:04.277937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:04.277950 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:04.278003 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:04.278986 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.279552 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.280544 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.280801 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.280874 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:04.283095 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:04.283097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:04.283097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:04.283105 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:04.284496 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:04.284880 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:04.284881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:04.284881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:04.284939 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:04.285161 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:04.285162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:04.285163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:04.285284 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:04.285342 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:04.285609 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 18:48:04.285703 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:48:04.285974 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:04.277034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45281 (local address 127.9.215.2:54061)
0811 18:48:04.277224 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:04.277283 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:04.277829 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:04.278074 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:04.278078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:04.278672 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:04.278880 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:04.278885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.279640 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.279641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:04.280458 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:04.280460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.280906 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.280907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:04.281085 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:04.281658 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:04.281667 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:04.283004 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:04.284575 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:04.284584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:04.284587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:04.284814 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:04.284994 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:04.284995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:04.284995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:04.285107 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:04.285338 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:04.285346 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:04.285493 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 18:48:04.285584 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:48:04.285755 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260811 18:48:04.286484 15588 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45281
I20260811 18:48:04.286607 15588 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:04.286823 15588 heartbeater.cc:511] Master 127.9.215.62:45281 requested a full tablet report, sending...
I20260811 18:48:04.287249 14931 ts_manager.cc:194] Registered new tserver with Master: 9197f8f2291c4192a9c8b98038e38e8a (127.9.215.2:36301)
I20260811 18:48:04.287672 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:54061
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:04.308063 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45281
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:04.391846 15594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:04.392087 15594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:04.392125 15594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:04.393381 15594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:04.393435 15594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:04.393474 15594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:04.395200 15594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:04.395500 15594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:04.395747 15594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:04.400950 15606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:04.400955 15611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:04.400950 15608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:04.401216 15594 server_base.cc:1034] running on GCE node
I20260811 18:48:04.401412 15594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:04.401710 15594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:04.402889 15594 hybrid_clock.cc:648] HybridClock initialized: now 1786474084402863 us; error 38 us; skew 500 ppm
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[14841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:04.405498 15594 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:04.405997 15594 webserver.cc:460] Webserver started at http://127.9.215.3:43715/ using document root <none> and password file <none>
I20260811 18:48:04.406245 15594 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:04.406298 15594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:04.406430 15594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:04.407636 15594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "1e5b34cab9294707ac97965e7d63e098"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "430e4f03fed6384fea39b8dd57b8302a"
server_key_iv: "e12359b3b55128257ee7cc596af2c770"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.408005 15594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "1e5b34cab9294707ac97965e7d63e098"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "430e4f03fed6384fea39b8dd57b8302a"
server_key_iv: "e12359b3b55128257ee7cc596af2c770"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.409346 15594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:04.410151 15628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.410359 15594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:04.410459 15594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "1e5b34cab9294707ac97965e7d63e098"
format_stamp: "Formatted at 2026-08-11 18:48:04 on dist-test-slave-pww0"
server_key: "430e4f03fed6384fea39b8dd57b8302a"
server_key_iv: "e12359b3b55128257ee7cc596af2c770"
server_key_version: "encryptionkey@0"
I20260811 18:48:04.410523 15594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:04.435375 15594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:04.437726 15594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:04.437896 15594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:04.438153 15594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:04.438496 15594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:04.438537 15594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.438567 15594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:04.438584 15594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:04.444700 15594 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:46391
I20260811 18:48:04.444753 15835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:46391 every 8 connection(s)
I20260811 18:48:04.445101 15594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[14841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:04.454579 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:04.445853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:53471 (local address 127.9.215.62:45281)
0811 18:48:04.445914 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:04.445916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:04.446128 (+ 212us) server_negotiation.cc:407] Connection header received
0811 18:48:04.446665 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:04.446667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:04.446679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:04.446735 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:04.447780 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.448280 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.449297 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.449489 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.449544 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:04.451816 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:04.451819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:04.451819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:04.451828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:04.453186 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:04.453619 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:04.453620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:04.453621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:04.453677 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:04.453915 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:04.453916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:04.453916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:04.454038 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:04.454100 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:04.454392 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 18:48:04.454484 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:48:04.454561 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15594
I20260811 18:48:04.454675 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:04.454778 15839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:04.445829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45281 (local address 127.9.215.3:53471)
0811 18:48:04.446002 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:04.446043 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:04.446563 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:04.446813 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:04.446817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:04.447418 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:04.447670 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:04.447676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.448369 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.448370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:04.449196 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:04.449198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:04.449560 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:04.449562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:04.449687 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:04.450208 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:04.450217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:04.451704 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:04.453285 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:04.453297 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:04.453299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:04.453543 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:04.453735 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:04.453736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:04.453736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:04.453855 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:04.454095 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:04.454103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:04.454262 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 18:48:04.454358 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:48:04.454534 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260811 18:48:04.454874 10076 external_mini_cluster.cc:1428] Setting key 69246529d4fc1265c01392f77d921a00
I20260811 18:48:04.455451 15837 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45281
I20260811 18:48:04.455586 15837 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:04.455801 15837 heartbeater.cc:511] Master 127.9.215.62:45281 requested a full tablet report, sending...
I20260811 18:48:04.456174 14928 ts_manager.cc:194] Registered new tserver with Master: 1e5b34cab9294707ac97965e7d63e098 (127.9.215.3:46391)
I20260811 18:48:04.456538 14928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:53471
I20260811 18:48:04.466815 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:48:04.485121 15014 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:04.483942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.9.215.62:45281)
0811 18:48:04.484017 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:04.484019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:04.484110 (+ 91us) server_negotiation.cc:407] Connection header received
0811 18:48:04.484186 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:04.484187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:04.484198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:04.484256 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:04.485034 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36798: BlockingRecv error: recv got EOF from 127.0.0.1:36798 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:48:04.485989 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15043
I20260811 18:48:04.508193 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15322
I20260811 18:48:04.529829 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15594
I20260811 18:48:04.542253 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 14868
2026-08-11T18:48:04Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2157 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[15864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[15864](info): set up 2 sockets
Aug 11 18:48:04 dist-test-slave-pww0 krb5kdc[15864](info): commencing operation
krb5kdc: starting...
W20260811 18:48:05.807621 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 18:48:05.979247 10076 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.172s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 18:48:06.114253 10076 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.135s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 18:48:06.228920 10076 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.115s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/test-user.keytab.
W20260811 18:48:06.462891 10076 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.234s user 0.000s sys 0.002s
Aug 11 18:48:06 dist-test-slave-pww0 krb5kdc[15864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:06.511097 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36013
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:06.594143 15887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:06.594376 15887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:06.594400 15887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:06.595629 15887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:06.595678 15887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:06.595691 15887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:06.595703 15887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:06.595714 15887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:06.597301 15887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:06.597565 15887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:06.597796 15887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:06.602988 15901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:06.603137 15887 server_base.cc:1034] running on GCE node
W20260811 18:48:06.602979 15898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:06.602979 15899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:06.603515 15887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:06.603822 15887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:06.604980 15887 hybrid_clock.cc:648] HybridClock initialized: now 1786474086604964 us; error 35 us; skew 500 ppm
Aug 11 18:48:06 dist-test-slave-pww0 krb5kdc[15864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474086, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:06.607597 15887 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:06.608095 15887 webserver.cc:460] Webserver started at http://127.9.215.62:44551/ using document root <none> and password file <none>
I20260811 18:48:06.608342 15887 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:06.608392 15887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:06.608539 15887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:06.609643 15887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "7e3768dc21024e07a59346a3e26e8fd2"
format_stamp: "Formatted at 2026-08-11 18:48:06 on dist-test-slave-pww0"
server_key: "fb01121e46c98bb6606af6ae052531eb"
server_key_iv: "1b1918a5790471545ba0410191f4dd5d"
server_key_version: "encryptionkey@0"
I20260811 18:48:06.609990 15887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "7e3768dc21024e07a59346a3e26e8fd2"
format_stamp: "Formatted at 2026-08-11 18:48:06 on dist-test-slave-pww0"
server_key: "fb01121e46c98bb6606af6ae052531eb"
server_key_iv: "1b1918a5790471545ba0410191f4dd5d"
server_key_version: "encryptionkey@0"
I20260811 18:48:06.611318 15887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:48:06.612067 15920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:06.612215 15887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:06.612296 15887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "7e3768dc21024e07a59346a3e26e8fd2"
format_stamp: "Formatted at 2026-08-11 18:48:06 on dist-test-slave-pww0"
server_key: "fb01121e46c98bb6606af6ae052531eb"
server_key_iv: "1b1918a5790471545ba0410191f4dd5d"
server_key_version: "encryptionkey@0"
I20260811 18:48:06.612367 15887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:06.644209 15887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:06.646579 15887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:06.646759 15887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:06.651077 15887 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36013
I20260811 18:48:06.651113 16039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36013 every 8 connection(s)
I20260811 18:48:06.651774 15887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:06.652518 16043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:06.655002 16043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2: Bootstrap starting.
I20260811 18:48:06.655802 16043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:06.656096 16043 log.cc:824] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:06.656857 16043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2: No bootstrap required, opened a new log
I20260811 18:48:06.657097 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15887
I20260811 18:48:06.657192 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:06.657354 10076 external_mini_cluster.cc:1428] Setting key d12b38346ce3a19c4a40dc842f0f1bc1
I20260811 18:48:06.659554 16043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3768dc21024e07a59346a3e26e8fd2" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36013 } }
I20260811 18:48:06.659756 16043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:06.659791 16043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e3768dc21024e07a59346a3e26e8fd2, State: Initialized, Role: FOLLOWER
I20260811 18:48:06.659976 16043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3768dc21024e07a59346a3e26e8fd2" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36013 } }
I20260811 18:48:06.660073 16043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:06.660110 16043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:06.660166 16043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:06.660971 16043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3768dc21024e07a59346a3e26e8fd2" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36013 } }
I20260811 18:48:06.661082 16043 leader_election.cc:302] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e3768dc21024e07a59346a3e26e8fd2; no voters:
I20260811 18:48:06.661270 16043 leader_election.cc:288] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:06.661342 16051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:06.661521 16051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 7e3768dc21024e07a59346a3e26e8fd2, State: Running, Role: LEADER
I20260811 18:48:06.661582 16043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:06.661644 16051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3768dc21024e07a59346a3e26e8fd2" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36013 } }
I20260811 18:48:06.662046 16055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e3768dc21024e07a59346a3e26e8fd2. Latest consensus state: current_term: 1 leader_uuid: "7e3768dc21024e07a59346a3e26e8fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3768dc21024e07a59346a3e26e8fd2" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36013 } } }
I20260811 18:48:06.662140 16055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:06.662305 16053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e3768dc21024e07a59346a3e26e8fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e3768dc21024e07a59346a3e26e8fd2" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36013 } } }
I20260811 18:48:06.662369 16053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:06.662348 16064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:06.662958 16064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:06.664644 16064 catalog_manager.cc:1347] Generated new cluster ID: ca84c6741992495f89cbdf73a2e3ef6d
I20260811 18:48:06.664698 16064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:48:06 dist-test-slave-pww0 krb5kdc[15864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:06.670976 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:06.657938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.9.215.62:36013)
0811 18:48:06.658127 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:06.658136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:06.658162 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:48:06.659558 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:06.659570 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:06.659645 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:06.659845 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:06.660687 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:06.661900 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:06.663028 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:06.663660 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:06.663768 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:06.666590 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:06.666595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:06.666596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:06.666610 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:06.668659 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:06.669059 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:06.669062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:06.669063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:06.669712 (+ 649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:06.669925 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:06.669926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:06.669927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:06.670310 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:06.670495 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:06.670570 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:48:06.670729 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260811 18:48:06.672325 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:48:06.679427 16064 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:06.680174 16064 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:06.690562 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:06.686039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.9.215.62:36013)
0811 18:48:06.686085 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:06.686087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:06.686258 (+ 171us) server_negotiation.cc:407] Connection header received
0811 18:48:06.686331 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:06.686332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:06.686347 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:06.686403 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:06.686885 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:06.687678 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:06.688799 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:06.689121 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:06.689178 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:06.689990 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:06.689992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:06.689992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:06.690288 (+ 296us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:06.690312 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:06.690320 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:06.690373 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:06.690437 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 18:48:06.690473 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:48:06.699120 16064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e3768dc21024e07a59346a3e26e8fd2: Generated new TSK 0
I20260811 18:48:06.699332 16064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:06.707211 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15887
2026-08-11T18:48:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2158 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:06 dist-test-slave-pww0 krb5kdc[16111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:06 dist-test-slave-pww0 krb5kdc[16111](info): set up 2 sockets
Aug 11 18:48:06 dist-test-slave-pww0 krb5kdc[16111](info): commencing operation
krb5kdc: starting...
W20260811 18:48:07.959436 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:08.091408 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36395
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:08.179124 16144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:08.179399 16144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:08.179436 16144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:08.180792 16144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:08.180855 16144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:08.180876 16144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:08.180891 16144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:08.180907 16144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:08.182615 16144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:08.182932 16144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:08.183167 16144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:08.188318 16152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:08.188424 16156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:08.188385 16153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:08.188476 16144 server_base.cc:1034] running on GCE node
I20260811 18:48:08.188776 16144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:08.189064 16144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:08.190201 16144 hybrid_clock.cc:648] HybridClock initialized: now 1786474088190183 us; error 36 us; skew 500 ppm
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:08.192955 16144 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:08.193538 16144 webserver.cc:460] Webserver started at http://127.9.215.62:45743/ using document root <none> and password file <none>
I20260811 18:48:08.193764 16144 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:08.193817 16144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:08.193949 16144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:08.195086 16144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "c0a3b0b0849d41eb896b713091f40e0a"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "e214faa710c1d549f76def428e29f2fc"
server_key_iv: "7bdafbe95379cd55ba82036c94d0612c"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.195487 16144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "c0a3b0b0849d41eb896b713091f40e0a"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "e214faa710c1d549f76def428e29f2fc"
server_key_iv: "7bdafbe95379cd55ba82036c94d0612c"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.196943 16144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:08.197753 16169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.197911 16144 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 18:48:08.198009 16144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c0a3b0b0849d41eb896b713091f40e0a"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "e214faa710c1d549f76def428e29f2fc"
server_key_iv: "7bdafbe95379cd55ba82036c94d0612c"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.198074 16144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:08.208302 16144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:08.210439 16144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:08.210594 16144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:08.214795 16144 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36395
I20260811 18:48:08.214836 16264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36395 every 8 connection(s)
I20260811 18:48:08.215209 16144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:08.216504 16267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:08.217473 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16144
I20260811 18:48:08.217576 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:08.217754 10076 external_mini_cluster.cc:1428] Setting key c83ed08d3aebff63dd47c568a403d8d6
I20260811 18:48:08.219954 16267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a: Bootstrap starting.
I20260811 18:48:08.220647 16267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:08.221086 16267 log.cc:824] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:08.221915 16267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a: No bootstrap required, opened a new log
I20260811 18:48:08.224414 16267 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3b0b0849d41eb896b713091f40e0a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36395 } }
I20260811 18:48:08.224591 16267 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:08.224617 16267 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a3b0b0849d41eb896b713091f40e0a, State: Initialized, Role: FOLLOWER
I20260811 18:48:08.224743 16267 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3b0b0849d41eb896b713091f40e0a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36395 } }
I20260811 18:48:08.224828 16267 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:08.224861 16267 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:08.224911 16267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:08.225713 16267 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3b0b0849d41eb896b713091f40e0a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36395 } }
I20260811 18:48:08.225845 16267 leader_election.cc:302] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0a3b0b0849d41eb896b713091f40e0a; no voters:
I20260811 18:48:08.226049 16267 leader_election.cc:288] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:08.226090 16279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:08.226250 16279 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [term 1 LEADER]: Becoming Leader. State: Replica: c0a3b0b0849d41eb896b713091f40e0a, State: Running, Role: LEADER
I20260811 18:48:08.226351 16279 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3b0b0849d41eb896b713091f40e0a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36395 } }
I20260811 18:48:08.226372 16267 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:08.226758 16280 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0a3b0b0849d41eb896b713091f40e0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3b0b0849d41eb896b713091f40e0a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36395 } } }
I20260811 18:48:08.226761 16282 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0a3b0b0849d41eb896b713091f40e0a. Latest consensus state: current_term: 1 leader_uuid: "c0a3b0b0849d41eb896b713091f40e0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3b0b0849d41eb896b713091f40e0a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36395 } } }
I20260811 18:48:08.226936 16280 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:08.227032 16282 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:08.227308 16298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:08.228222 16298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:08.230101 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.218315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.9.215.62:36395)
0811 18:48:08.218478 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:08.218487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:08.218524 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:48:08.218958 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:08.218964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:08.219016 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:08.219145 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:08.219923 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.220918 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.221950 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.222386 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.222468 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.225165 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:08.225170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:08.225171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:08.225184 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:08.228302 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.228774 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.228778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.228779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.228898 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.229155 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.229155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.229156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.229485 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:08.229640 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:08.229745 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 18:48:08.229895 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260811 18:48:08.232684 16298 catalog_manager.cc:1347] Generated new cluster ID: 5b84bc9141554f479a36a9cb665b3148
I20260811 18:48:08.232745 16298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:08.256740 16298 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:08.257512 16298 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:08.266736 16298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0a3b0b0849d41eb896b713091f40e0a: Generated new TSK 0
I20260811 18:48:08.266995 16298 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:08.300104 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36395
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:08.393428 16331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:08.393656 16331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:08.393680 16331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:08.394872 16331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:08.394922 16331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:08.394949 16331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:08.396482 16331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:08.396726 16331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:08.396948 16331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:08.402176 16344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:08.402191 16342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:08.402187 16347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:08.402290 16331 server_base.cc:1034] running on GCE node
I20260811 18:48:08.402591 16331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:08.402842 16331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:08.404017 16331 hybrid_clock.cc:648] HybridClock initialized: now 1786474088403981 us; error 57 us; skew 500 ppm
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:08.406627 16331 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:08.407116 16331 webserver.cc:460] Webserver started at http://127.9.215.1:35565/ using document root <none> and password file <none>
I20260811 18:48:08.407322 16331 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:08.407364 16331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:08.407509 16331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:08.408684 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "470d9bfdc6be4fb9a51e9578b03c9155"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "14977e1e5e715a59accfe2eda387d502"
server_key_iv: "7d0fc167268e2e570407ab7d2c4d1be6"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.409034 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "470d9bfdc6be4fb9a51e9578b03c9155"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "14977e1e5e715a59accfe2eda387d502"
server_key_iv: "7d0fc167268e2e570407ab7d2c4d1be6"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.410410 16331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:08.411271 16356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.411437 16331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:08.411552 16331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "470d9bfdc6be4fb9a51e9578b03c9155"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "14977e1e5e715a59accfe2eda387d502"
server_key_iv: "7d0fc167268e2e570407ab7d2c4d1be6"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.411620 16331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:08.429417 16331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:08.431775 16331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:08.431905 16331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:08.432123 16331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:08.432425 16331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:08.432459 16331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.432482 16331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:08.432498 16331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.438412 16331 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:39487
I20260811 18:48:08.438488 16599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:39487 every 8 connection(s)
I20260811 18:48:08.438817 16331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:08.448204 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16331
I20260811 18:48:08.448319 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:08.448525 10076 external_mini_cluster.cc:1428] Setting key 3ebd5434745b707386e5c8c789adff28
I20260811 18:48:08.448549 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.439593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:38805 (local address 127.9.215.62:36395)
0811 18:48:08.439671 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:08.439673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:08.439936 (+ 263us) server_negotiation.cc:407] Connection header received
0811 18:48:08.440484 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:08.440485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:08.440497 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:08.440553 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:08.441576 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.442097 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.443115 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.443321 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.443381 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.445756 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:08.445758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:08.445758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:08.445766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:08.446981 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.447410 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.447412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.447412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.447559 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.447793 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.447794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.447795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.447928 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:08.447986 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:08.448371 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:48:08.448475 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:08.448802 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.439626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36395 (local address 127.9.215.1:38805)
0811 18:48:08.439800 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:08.439841 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:08.440380 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:08.440646 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:08.440650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:08.441262 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:08.441469 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:08.441473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.442191 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.442193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:08.443024 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:08.443027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.443401 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.443402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:08.443588 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.444131 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:08.444152 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:08.445649 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:08.447058 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:08.447065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:08.447068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:08.447316 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:08.447623 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:08.447623 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:08.447624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:08.447731 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:08.447986 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:08.447991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:08.448192 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 18:48:08.448311 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 18:48:08.448496 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260811 18:48:08.449365 16604 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36395
I20260811 18:48:08.449462 16604 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:08.449654 16604 heartbeater.cc:511] Master 127.9.215.62:36395 requested a full tablet report, sending...
I20260811 18:48:08.450290 16202 ts_manager.cc:194] Registered new tserver with Master: 470d9bfdc6be4fb9a51e9578b03c9155 (127.9.215.1:39487)
I20260811 18:48:08.451162 16202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:38805
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:08.478060 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36395
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:08.563915 16613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:08.564143 16613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:08.564181 16613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:08.565471 16613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:08.565523 16613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:08.565564 16613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:08.567193 16613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:08.567505 16613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:08.567765 16613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:08.572949 16619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:08.573112 16613 server_base.cc:1034] running on GCE node
W20260811 18:48:08.572952 16620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:08.572952 16626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:08.573410 16613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:08.573693 16613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:08.574846 16613 hybrid_clock.cc:648] HybridClock initialized: now 1786474088574831 us; error 31 us; skew 500 ppm
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:08.577670 16613 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:08.578186 16613 webserver.cc:460] Webserver started at http://127.9.215.2:40711/ using document root <none> and password file <none>
I20260811 18:48:08.578431 16613 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:08.578483 16613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:08.578608 16613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:08.579767 16613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "1deee581c3f04228b1c34fa4e8c3029f"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "601f80c7e65fd91ccbe85c967b9c080c"
server_key_iv: "f37872e7dad1fe072f1a54136126531e"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.580125 16613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "1deee581c3f04228b1c34fa4e8c3029f"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "601f80c7e65fd91ccbe85c967b9c080c"
server_key_iv: "f37872e7dad1fe072f1a54136126531e"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.581570 16613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:08.582366 16637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.582513 16613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:08.582607 16613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "1deee581c3f04228b1c34fa4e8c3029f"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "601f80c7e65fd91ccbe85c967b9c080c"
server_key_iv: "f37872e7dad1fe072f1a54136126531e"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.582674 16613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:08.607947 16613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:08.610149 16613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:08.610337 16613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:08.610587 16613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:08.610914 16613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:08.610950 16613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.610980 16613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:08.611004 16613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.617029 16613 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:37085
I20260811 18:48:08.617091 16876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:37085 every 8 connection(s)
I20260811 18:48:08.617429 16613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:08.624363 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16613
I20260811 18:48:08.624469 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:08.624719 10076 external_mini_cluster.cc:1428] Setting key 4a35aaedcc75f336e1c276bc51b62226
I20260811 18:48:08.627018 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.618175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:40513 (local address 127.9.215.62:36395)
0811 18:48:08.618229 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:08.618231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:08.618439 (+ 208us) server_negotiation.cc:407] Connection header received
0811 18:48:08.618934 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:08.618935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:08.618947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:08.619002 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:08.620012 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.620530 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.621537 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.621739 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.621798 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.623999 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:08.624001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:08.624002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:08.624011 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:08.625629 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.626069 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.626071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.626071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.626135 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.626372 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.626373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.626373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.626506 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:08.626562 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:08.626842 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:48:08.626925 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:48:08.627163 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.618162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36395 (local address 127.9.215.2:40513)
0811 18:48:08.618317 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:08.618357 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:08.618846 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:08.619058 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:08.619062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:08.619709 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:08.619923 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:08.619928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.620605 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.620606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:08.621450 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:08.621453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.621807 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.621808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:08.621914 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.622438 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:08.622446 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:08.623900 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:08.625699 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:08.625711 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:08.625714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:08.625994 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:08.626194 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:08.626195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:08.626195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:08.626319 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:08.626560 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:08.626563 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:08.626707 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 18:48:08.626794 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:48:08.626958 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260811 18:48:08.627822 16877 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36395
I20260811 18:48:08.627933 16877 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:08.628131 16877 heartbeater.cc:511] Master 127.9.215.62:36395 requested a full tablet report, sending...
I20260811 18:48:08.628543 16202 ts_manager.cc:194] Registered new tserver with Master: 1deee581c3f04228b1c34fa4e8c3029f (127.9.215.2:37085)
I20260811 18:48:08.629011 16202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:40513
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:08.654083 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36395
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:08.740415 16885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:08.740661 16885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:08.740700 16885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:08.742002 16885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:08.742060 16885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:08.742108 16885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:08.743793 16885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:08.744102 16885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:08.744343 16885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:08.749626 16894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:08.749626 16897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:08.749728 16885 server_base.cc:1034] running on GCE node
W20260811 18:48:08.749776 16900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:08.750123 16885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:08.750396 16885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:08.751541 16885 hybrid_clock.cc:648] HybridClock initialized: now 1786474088751525 us; error 32 us; skew 500 ppm
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:08.754187 16885 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:08.754698 16885 webserver.cc:460] Webserver started at http://127.9.215.3:46441/ using document root <none> and password file <none>
I20260811 18:48:08.754932 16885 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:08.754985 16885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:08.755128 16885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:08.756276 16885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "8b5a3dc2fae0493bba7d5b4df662b2e8"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "cc543b7ac90b65f182725b3d81be08d0"
server_key_iv: "54916e6267de6ffa1c3733c34a91595d"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.756629 16885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "8b5a3dc2fae0493bba7d5b4df662b2e8"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "cc543b7ac90b65f182725b3d81be08d0"
server_key_iv: "54916e6267de6ffa1c3733c34a91595d"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.758028 16885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:08.758780 16912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.758955 16885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 18:48:08.759050 16885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "8b5a3dc2fae0493bba7d5b4df662b2e8"
format_stamp: "Formatted at 2026-08-11 18:48:08 on dist-test-slave-pww0"
server_key: "cc543b7ac90b65f182725b3d81be08d0"
server_key_iv: "54916e6267de6ffa1c3733c34a91595d"
server_key_version: "encryptionkey@0"
I20260811 18:48:08.759130 16885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:08.787348 16885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:08.789283 16885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:08.789434 16885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:08.789675 16885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:08.789994 16885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:08.790027 16885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.790057 16885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:08.790081 16885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:08.796167 16885 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:38727
I20260811 18:48:08.796206 17136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:38727 every 8 connection(s)
I20260811 18:48:08.796545 16885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:08.800539 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16885
I20260811 18:48:08.800634 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:08.800834 10076 external_mini_cluster.cc:1428] Setting key e67e1150e3214fdba8587117ab9422fa
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:08.806125 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.797259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:52845 (local address 127.9.215.62:36395)
0811 18:48:08.797335 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:08.797337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:08.797501 (+ 164us) server_negotiation.cc:407] Connection header received
0811 18:48:08.797989 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:08.797991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:08.798001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:08.798051 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:08.798993 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.799583 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.800769 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.801016 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.801084 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.803380 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:08.803382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:08.803382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:08.803389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:08.804773 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.805178 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.805179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.805180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.805235 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.805480 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.805481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.805482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.805600 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:08.805656 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:08.805942 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 18:48:08.806031 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:48:08.806322 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.797219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36395 (local address 127.9.215.3:52845)
0811 18:48:08.797381 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:08.797419 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:08.797906 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:08.798115 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:08.798119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:08.798692 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:08.798902 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:08.798908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.799653 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.799654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:08.800637 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:08.800641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.801100 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.801101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:08.801287 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.801842 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:08.801851 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:08.803289 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:08.804864 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:08.804873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:08.804876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:08.805107 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:08.805297 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:08.805298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:08.805299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:08.805423 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:08.805656 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:08.805664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:08.805819 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 18:48:08.805913 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:48:08.806087 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260811 18:48:08.806773 17137 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36395
I20260811 18:48:08.806872 17137 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:08.807057 17137 heartbeater.cc:511] Master 127.9.215.62:36395 requested a full tablet report, sending...
I20260811 18:48:08.807448 16202 ts_manager.cc:194] Registered new tserver with Master: 8b5a3dc2fae0493bba7d5b4df662b2e8 (127.9.215.3:38727)
I20260811 18:48:08.807874 16202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:52845
I20260811 18:48:08.812729 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[16111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474088, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:08.836905 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:08.829965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.9.215.62:36395)
0811 18:48:08.830019 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:08.830020 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:08.830098 (+ 78us) server_negotiation.cc:407] Connection header received
0811 18:48:08.830168 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:08.830169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:08.830180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:08.830224 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:08.830835 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.831368 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.832104 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:08.832299 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:08.832357 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:08.834553 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:08.834555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:08.834555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:08.834564 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:08.835935 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.836244 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.836245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.836246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.836302 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:08.836437 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:08.836438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:08.836438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:08.836597 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:08.836675 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:08.836760 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 18:48:08.836829 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 18:48:08.837153 16202 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60306
I20260811 18:48:08.837631 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16331
I20260811 18:48:08.842846 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16613
I20260811 18:48:08.876118 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16885
I20260811 18:48:08.893507 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16144
2026-08-11T18:48:08Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2193 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[17169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[17169](info): set up 2 sockets
Aug 11 18:48:08 dist-test-slave-pww0 krb5kdc[17169](info): commencing operation
krb5kdc: starting...
W20260811 18:48:10.141109 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:10.267273 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:44891
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:44891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:10.352684 17193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:10.352905 17193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:10.352931 17193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:10.354215 17193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:10.354275 17193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:10.354290 17193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:10.354301 17193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:10.354311 17193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:10.356025 17193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:44891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:44891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:10.356302 17193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:10.356544 17193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:10.361697 17205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:10.361697 17202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:10.361904 17193 server_base.cc:1034] running on GCE node
W20260811 18:48:10.361793 17201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:10.362221 17193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:10.362603 17193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:10.363750 17193 hybrid_clock.cc:648] HybridClock initialized: now 1786474090363737 us; error 33 us; skew 500 ppm
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:10.366307 17193 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:10.366765 17193 webserver.cc:460] Webserver started at http://127.9.215.62:36317/ using document root <none> and password file <none>
I20260811 18:48:10.366989 17193 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:10.367039 17193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:10.367172 17193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:10.368306 17193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "736b4965d833491791f22f692308ed8a"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "453ec6daad75514cdf73d858d3305906"
server_key_iv: "09c5b6a725ac06551aa90dcda8819eea"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.368637 17193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "736b4965d833491791f22f692308ed8a"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "453ec6daad75514cdf73d858d3305906"
server_key_iv: "09c5b6a725ac06551aa90dcda8819eea"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.370015 17193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:10.370780 17219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.370995 17193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:10.371114 17193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "736b4965d833491791f22f692308ed8a"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "453ec6daad75514cdf73d858d3305906"
server_key_iv: "09c5b6a725ac06551aa90dcda8819eea"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.371182 17193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:10.395609 17193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:10.397940 17193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:10.398106 17193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:10.402441 17193 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:44891
I20260811 18:48:10.402508 17320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:44891 every 8 connection(s)
I20260811 18:48:10.402994 17193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:10.403579 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17193
I20260811 18:48:10.403684 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:10.403887 10076 external_mini_cluster.cc:1428] Setting key 6f14ecf0875f7b66f559f272f91a732c
I20260811 18:48:10.403931 17323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:10.407205 17323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a: Bootstrap starting.
I20260811 18:48:10.408026 17323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:10.408360 17323 log.cc:824] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:10.409165 17323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a: No bootstrap required, opened a new log
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:10.411158 17323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b4965d833491791f22f692308ed8a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44891 } }
I20260811 18:48:10.411355 17323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:10.411397 17323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736b4965d833491791f22f692308ed8a, State: Initialized, Role: FOLLOWER
I20260811 18:48:10.411545 17323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b4965d833491791f22f692308ed8a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44891 } }
I20260811 18:48:10.411617 17323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:10.411648 17323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:10.411688 17323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:10.412403 17323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b4965d833491791f22f692308ed8a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44891 } }
I20260811 18:48:10.412519 17323 leader_election.cc:302] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 736b4965d833491791f22f692308ed8a; no voters:
I20260811 18:48:10.412711 17323 leader_election.cc:288] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:10.412772 17336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:10.412994 17336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [term 1 LEADER]: Becoming Leader. State: Replica: 736b4965d833491791f22f692308ed8a, State: Running, Role: LEADER
I20260811 18:48:10.413116 17323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:10.413151 17336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b4965d833491791f22f692308ed8a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44891 } }
I20260811 18:48:10.413520 17337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "736b4965d833491791f22f692308ed8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b4965d833491791f22f692308ed8a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44891 } } }
I20260811 18:48:10.413601 17337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:10.413688 17340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 736b4965d833491791f22f692308ed8a. Latest consensus state: current_term: 1 leader_uuid: "736b4965d833491791f22f692308ed8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "736b4965d833491791f22f692308ed8a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44891 } } }
I20260811 18:48:10.413750 17340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:10.414196 17351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:10.414739 17351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:10.415449 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.404449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.9.215.62:44891)
0811 18:48:10.404621 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:10.404631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:10.404657 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:48:10.405069 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:10.405076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:10.405125 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:10.405241 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:10.406030 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.407043 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.408097 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.408615 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.408709 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.411056 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:10.411061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:10.411062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:10.411076 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:10.413120 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.413583 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.413586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.413587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.413976 (+ 389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.414235 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.414236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.414236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.414581 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:10.414760 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:10.414919 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 18:48:10.415167 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260811 18:48:10.416509 17351 catalog_manager.cc:1347] Generated new cluster ID: 77a2b60a651b45f19b8ce66514f5980e
I20260811 18:48:10.416558 17351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:10.433171 17351 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:10.433969 17351 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:10.441190 17351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 736b4965d833491791f22f692308ed8a: Generated new TSK 0
I20260811 18:48:10.441439 17351 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:10.472426 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44891
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:10.557670 17378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:10.557946 17378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:10.557983 17378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:10.559253 17378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:10.559305 17378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:10.559343 17378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:10.561094 17378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:10.561370 17378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:10.561599 17378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:10.566865 17403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:10.566907 17409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:10.566865 17405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:10.567087 17378 server_base.cc:1034] running on GCE node
I20260811 18:48:10.567323 17378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:10.567620 17378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:10.568795 17378 hybrid_clock.cc:648] HybridClock initialized: now 1786474090568773 us; error 38 us; skew 500 ppm
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:10.571497 17378 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:10.571981 17378 webserver.cc:460] Webserver started at http://127.9.215.1:41031/ using document root <none> and password file <none>
I20260811 18:48:10.572192 17378 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:10.572244 17378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:10.572343 17378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:10.573447 17378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "7ecd5b6ea7d04b1fad3e122951fca9c1"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "ae02feaf1261d7392dc80c533f779782"
server_key_iv: "65f1ce4eca149c989de1d0a5cc46bbab"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.573800 17378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "7ecd5b6ea7d04b1fad3e122951fca9c1"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "ae02feaf1261d7392dc80c533f779782"
server_key_iv: "65f1ce4eca149c989de1d0a5cc46bbab"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.575222 17378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:48:10.576109 17425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.576285 17378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:48:10.576381 17378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "7ecd5b6ea7d04b1fad3e122951fca9c1"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "ae02feaf1261d7392dc80c533f779782"
server_key_iv: "65f1ce4eca149c989de1d0a5cc46bbab"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.576452 17378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:10.589323 17378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:10.591610 17378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:10.591735 17378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:10.591945 17378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:10.592231 17378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:10.592262 17378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.592293 17378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:10.592310 17378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.598004 17378 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:43995
I20260811 18:48:10.598049 17651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:43995 every 8 connection(s)
I20260811 18:48:10.598389 17378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:10.598789 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17378
I20260811 18:48:10.598881 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:10.599081 10076 external_mini_cluster.cc:1428] Setting key 8428d485384bfd1307e22679155dbda8
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:48:10.609375 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.599185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:37005 (local address 127.9.215.62:44891)
0811 18:48:10.599235 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:10.599237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:10.599532 (+ 295us) server_negotiation.cc:407] Connection header received
0811 18:48:10.600734 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:10.600735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:10.600746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:10.600802 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:10.601681 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.602429 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.603761 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.603970 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.604027 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.606579 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:10.606581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:10.606581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:10.606588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:10.607918 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.608397 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.608398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.608398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.608454 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.608770 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.608771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.608771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.608898 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:10.608954 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:10.609233 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 18:48:10.609303 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:48:10.609618 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.599190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44891 (local address 127.9.215.1:37005)
0811 18:48:10.599381 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:10.599426 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:10.600601 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:10.600893 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:10.600898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:10.601254 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:10.601539 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:10.601546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.602540 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.602541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:10.603676 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:10.603680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.604056 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.604058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:10.604221 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.604779 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:10.604791 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:10.606469 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:10.608000 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:10.608009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:10.608012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:10.608308 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:10.608507 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:10.608508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:10.608509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:10.608660 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:10.608966 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:10.608975 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:10.609120 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 18:48:10.609211 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:48:10.609342 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260811 18:48:10.610132 17652 heartbeater.cc:347] Connected to a master server at 127.9.215.62:44891
I20260811 18:48:10.610229 17652 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:10.610419 17652 heartbeater.cc:511] Master 127.9.215.62:44891 requested a full tablet report, sending...
I20260811 18:48:10.611003 17243 ts_manager.cc:194] Registered new tserver with Master: 7ecd5b6ea7d04b1fad3e122951fca9c1 (127.9.215.1:43995)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
I20260811 18:48:10.611739 17243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:37005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:10.626297 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44891
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:10.711580 17661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:10.711807 17661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:10.711835 17661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:10.713132 17661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:10.713186 17661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:10.713217 17661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:10.714969 17661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:10.715195 17661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:10.715406 17661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:10.720610 17672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:10.720644 17675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:10.720649 17670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:10.720937 17661 server_base.cc:1034] running on GCE node
I20260811 18:48:10.721129 17661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:10.721410 17661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:10.722597 17661 hybrid_clock.cc:648] HybridClock initialized: now 1786474090722578 us; error 31 us; skew 500 ppm
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:10.725356 17661 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:10.725878 17661 webserver.cc:460] Webserver started at http://127.9.215.2:44277/ using document root <none> and password file <none>
I20260811 18:48:10.726119 17661 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:10.726166 17661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:10.726307 17661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:10.727391 17661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "f3b82e06caf04e82892ec0b7aa15ec1b"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "4706e6bb5a12975814df62745a402a41"
server_key_iv: "f2efa8d44638e41fd5877124ef198c03"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.727778 17661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "f3b82e06caf04e82892ec0b7aa15ec1b"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "4706e6bb5a12975814df62745a402a41"
server_key_iv: "f2efa8d44638e41fd5877124ef198c03"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.729132 17661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:10.729985 17691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.730166 17661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:10.730281 17661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "f3b82e06caf04e82892ec0b7aa15ec1b"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "4706e6bb5a12975814df62745a402a41"
server_key_iv: "f2efa8d44638e41fd5877124ef198c03"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.730363 17661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:10.740780 17661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:10.743057 17661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:10.743185 17661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:10.743443 17661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:10.743818 17661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:10.743855 17661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.743880 17661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:10.743903 17661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.749749 17661 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:36279
I20260811 18:48:10.749811 17918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:36279 every 8 connection(s)
I20260811 18:48:10.750136 17661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:10.752202 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17661
I20260811 18:48:10.752321 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:10.752526 10076 external_mini_cluster.cc:1428] Setting key 6d2ccc917038bd723ef5485e706a006b
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:10.760905 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.750950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:54275 (local address 127.9.215.62:44891)
0811 18:48:10.751007 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:10.751010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:10.751214 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:48:10.752234 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:10.752236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:10.752250 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:10.752304 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:10.753121 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.753821 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.754875 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.755146 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.755213 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.757769 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:10.757771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:10.757772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:10.757782 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:10.759338 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.759825 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.759827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.759827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.759897 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.760170 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.760170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.760171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.760329 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:10.760397 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:10.760713 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 18:48:10.760805 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:48:10.760973 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.750885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44891 (local address 127.9.215.2:54275)
0811 18:48:10.751083 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:10.751122 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:10.752119 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:10.752382 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:10.752386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:10.752743 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:10.753015 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:10.753022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.753931 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.753932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:10.754787 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:10.754789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.755247 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.755249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:10.755409 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.756009 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:10.756020 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:10.757655 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:10.759433 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:10.759442 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:10.759444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:10.759750 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:10.759972 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:10.759973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:10.759973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:10.760098 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:10.760397 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:10.760404 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:10.760549 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 18:48:10.760656 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:48:10.760792 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":84,"threads_started":1}
I20260811 18:48:10.761471 17920 heartbeater.cc:347] Connected to a master server at 127.9.215.62:44891
I20260811 18:48:10.761590 17920 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:10.761818 17920 heartbeater.cc:511] Master 127.9.215.62:44891 requested a full tablet report, sending...
I20260811 18:48:10.762223 17243 ts_manager.cc:194] Registered new tserver with Master: f3b82e06caf04e82892ec0b7aa15ec1b (127.9.215.2:36279)
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
I20260811 18:48:10.762588 17243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:54275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:10.782812 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44891
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:10.868561 17926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:10.868795 17926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:10.868821 17926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:10.870077 17926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:10.870129 17926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:10.870158 17926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:10.871843 17926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:10.872105 17926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:10.872350 17926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:10.877776 17941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:10.877923 17926 server_base.cc:1034] running on GCE node
W20260811 18:48:10.877743 17936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:10.877785 17933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:10.878271 17926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:10.878576 17926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:10.879721 17926 hybrid_clock.cc:648] HybridClock initialized: now 1786474090879710 us; error 41 us; skew 500 ppm
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:10.882488 17926 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:10.883015 17926 webserver.cc:460] Webserver started at http://127.9.215.3:42627/ using document root <none> and password file <none>
I20260811 18:48:10.883248 17926 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:10.883308 17926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:10.883445 17926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:10.884608 17926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "865c27e7ec9448b690e50425438c3f8a"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "bf4b98f5c22efdb214a428219555bf1c"
server_key_iv: "684b9e72f104e2f765561a4cf5ef5844"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.884984 17926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "865c27e7ec9448b690e50425438c3f8a"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "bf4b98f5c22efdb214a428219555bf1c"
server_key_iv: "684b9e72f104e2f765561a4cf5ef5844"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.886404 17926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:10.887319 17950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.887589 17926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:10.887710 17926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "865c27e7ec9448b690e50425438c3f8a"
format_stamp: "Formatted at 2026-08-11 18:48:10 on dist-test-slave-pww0"
server_key: "bf4b98f5c22efdb214a428219555bf1c"
server_key_iv: "684b9e72f104e2f765561a4cf5ef5844"
server_key_version: "encryptionkey@0"
I20260811 18:48:10.887789 17926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:10.918882 17926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:10.921641 17926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:10.921844 17926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:10.922109 17926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:10.922446 17926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:10.922483 17926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.922513 17926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:10.922538 17926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:10.928872 17926 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:35405
I20260811 18:48:10.928961 18165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:35405 every 8 connection(s)
I20260811 18:48:10.929291 17926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:10.939487 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17926
I20260811 18:48:10.939602 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:10.939790 10076 external_mini_cluster.cc:1428] Setting key 9561b2dfe804d7983e8e020bbf7f9536
I20260811 18:48:10.940150 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.930083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:34377 (local address 127.9.215.62:44891)
0811 18:48:10.930147 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:10.930149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:10.930394 (+ 245us) server_negotiation.cc:407] Connection header received
0811 18:48:10.931320 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:10.931322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:10.931334 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:10.931403 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:10.932143 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.932673 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.933894 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.934116 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.934184 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.936706 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:10.936708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:10.936709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:10.936719 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:10.938273 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.938815 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.938816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.938817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.938891 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.939268 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.939269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.939270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.939451 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:10.939561 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:10.939946 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:48:10.940047 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:48:10.940387 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.930048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44891 (local address 127.9.215.3:34377)
0811 18:48:10.930218 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:10.930282 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:10.931193 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:10.931495 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:10.931500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:10.931799 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:10.932013 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:10.932018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.932795 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.932796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:10.933793 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:10.933797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.934211 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.934212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:10.934338 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.934909 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:10.934918 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:10.936555 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:10.938391 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:10.938405 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:10.938407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:10.938710 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:10.938985 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:10.938986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:10.938987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:10.939143 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:10.939582 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:10.939595 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:10.939779 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 18:48:10.939907 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:48:10.940130 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260811 18:48:10.940944 18168 heartbeater.cc:347] Connected to a master server at 127.9.215.62:44891
I20260811 18:48:10.941056 18168 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:10.941293 18168 heartbeater.cc:511] Master 127.9.215.62:44891 requested a full tablet report, sending...
I20260811 18:48:10.941706 17243 ts_manager.cc:194] Registered new tserver with Master: 865c27e7ec9448b690e50425438c3f8a (127.9.215.3:35405)
I20260811 18:48:10.942093 17243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:34377
I20260811 18:48:10.952538 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:48:10.978726 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.969948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.9.215.1:43995)
0811 18:48:10.970178 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:10.970184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:10.970199 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:48:10.970226 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:10.970228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:10.970243 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:10.970333 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:10.970911 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.971795 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.972570 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.972943 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.973000 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.975526 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:10.975538 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:10.975539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:10.975547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:10.977223 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.977657 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.977661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.977661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.977736 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.978031 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.978033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.978033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.978232 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:10.978390 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:10.978506 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 18:48:10.978583 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
Aug 11 18:48:10 dist-test-slave-pww0 krb5kdc[17169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:10.987612 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:10.980134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.9.215.62:44891)
0811 18:48:10.980201 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:10.980203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:10.980213 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:10.980255 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:10.980256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:10.980266 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:10.980309 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:10.980787 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.981309 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.982058 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:10.982277 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:10.982337 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:10.984776 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:10.984778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:10.984778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:10.984787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:10.986428 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.986820 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.986821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.986822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.986892 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:10.987053 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:10.987053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:10.987054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:10.987212 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:10.987293 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:10.987439 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:48:10.987539 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 18:48:10.987968 17243 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53766
I20260811 18:48:10.988336 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17378
I20260811 18:48:11.011341 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17661
I20260811 18:48:11.021850 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17926
I20260811 18:48:11.045138 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17193
2026-08-11T18:48:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2151 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:11 dist-test-slave-pww0 krb5kdc[18194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:11 dist-test-slave-pww0 krb5kdc[18194](info): set up 2 sockets
Aug 11 18:48:11 dist-test-slave-pww0 krb5kdc[18194](info): commencing operation
krb5kdc: starting...
W20260811 18:48:12.277513 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:12.414743 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36683
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:12.504551 18215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:12.504956 18215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:12.505030 18215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:12.506868 18215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:12.507009 18215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:12.507071 18215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:12.507134 18215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:12.507187 18215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:12.509714 18215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:12.510181 18215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:12.510524 18215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:12.516719 18220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:12.517050 18224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:12.516719 18226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:12.525786 18215 server_base.cc:1034] running on GCE node
I20260811 18:48:12.526310 18215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:12.528329 18215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:12.529531 18215 hybrid_clock.cc:648] HybridClock initialized: now 1786474092529493 us; error 65 us; skew 500 ppm
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:12.532887 18215 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:12.533406 18215 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:12.533543 18215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:12.533707 18215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:12.535231 18215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "55fc8d03d97a49198c570852f06dd71a"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "bc547de4681aa2a328d246f97b189c34"
server_key_iv: "72cbca173ee50dec864940617265e93e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.535717 18215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "55fc8d03d97a49198c570852f06dd71a"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "bc547de4681aa2a328d246f97b189c34"
server_key_iv: "72cbca173ee50dec864940617265e93e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.537624 18215 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:48:12.538664 18236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.538902 18215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:12.539021 18215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "55fc8d03d97a49198c570852f06dd71a"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "bc547de4681aa2a328d246f97b189c34"
server_key_iv: "72cbca173ee50dec864940617265e93e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.539137 18215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:12.563336 18215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:12.565773 18215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:12.565933 18215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:12.570900 18215 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36683
I20260811 18:48:12.570932 18327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36683 every 8 connection(s)
I20260811 18:48:12.571285 18215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:12.572439 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18215
I20260811 18:48:12.572551 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:12.572748 10076 external_mini_cluster.cc:1428] Setting key 967e57ce4230888902f86cd35132b61e
I20260811 18:48:12.572919 18328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:12.577243 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a: Bootstrap starting.
I20260811 18:48:12.578084 18328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:12.578424 18328 log.cc:824] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:12.579195 18328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a: No bootstrap required, opened a new log
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:12.581527 18328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fc8d03d97a49198c570852f06dd71a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36683 } }
I20260811 18:48:12.581743 18328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:12.581789 18328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55fc8d03d97a49198c570852f06dd71a, State: Initialized, Role: FOLLOWER
I20260811 18:48:12.581933 18328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fc8d03d97a49198c570852f06dd71a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36683 } }
I20260811 18:48:12.582022 18328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:12.582062 18328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:12.582103 18328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:12.582942 18328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fc8d03d97a49198c570852f06dd71a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36683 } }
I20260811 18:48:12.583055 18328 leader_election.cc:302] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55fc8d03d97a49198c570852f06dd71a; no voters:
I20260811 18:48:12.583283 18328 leader_election.cc:288] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:12.583355 18334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:12.583568 18334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [term 1 LEADER]: Becoming Leader. State: Replica: 55fc8d03d97a49198c570852f06dd71a, State: Running, Role: LEADER
I20260811 18:48:12.583626 18328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:12.583666 18334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fc8d03d97a49198c570852f06dd71a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36683 } }
I20260811 18:48:12.584014 18334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55fc8d03d97a49198c570852f06dd71a. Latest consensus state: current_term: 1 leader_uuid: "55fc8d03d97a49198c570852f06dd71a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fc8d03d97a49198c570852f06dd71a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36683 } } }
I20260811 18:48:12.584022 18335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55fc8d03d97a49198c570852f06dd71a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55fc8d03d97a49198c570852f06dd71a" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36683 } } }
I20260811 18:48:12.584113 18334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:12.584116 18335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:12.584849 18346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:12.585412 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:12.573367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.9.215.62:36683)
0811 18:48:12.573580 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:12.573588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:12.573620 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:48:12.574115 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:12.574122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:12.574180 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:12.574329 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:12.574982 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.576005 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.576827 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.577265 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.577365 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:12.580327 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:12.580332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:12.580333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:12.580348 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:12.582787 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:12.583250 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:12.583255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:12.583255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:12.583561 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:12.583986 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:12.583987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:12.583987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:12.584471 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:12.584635 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:12.585037 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 18:48:12.585197 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":98,"threads_started":1}
I20260811 18:48:12.585809 18346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:12.587613 18346 catalog_manager.cc:1347] Generated new cluster ID: 26922b4a2f5943a7a87febc550480525
I20260811 18:48:12.587675 18346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:12.602964 18346 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:12.603808 18346 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:12.612424 18346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a: Generated new TSK 0
I20260811 18:48:12.612694 18346 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:12.651274 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36683
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:12.756125 18374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:12.756362 18374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:12.756398 18374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:12.757655 18374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:12.757720 18374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:12.757761 18374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:12.759898 18374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:12.760215 18374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:12.760469 18374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:12.765856 18392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:12.765842 18388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:12.765898 18374 server_base.cc:1034] running on GCE node
W20260811 18:48:12.765842 18389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:12.766376 18374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:12.766649 18374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:12.767812 18374 hybrid_clock.cc:648] HybridClock initialized: now 1786474092767793 us; error 31 us; skew 500 ppm
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:12.770538 18374 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:12.770936 18374 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:12.770988 18374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:12.771091 18374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:12.772222 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "667ec5db9c0a4b8eb830e195f5a13b88"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "3611751a0721d15b30859d93ad040d12"
server_key_iv: "5636610ee966420a3ea017f56979316e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.772524 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "667ec5db9c0a4b8eb830e195f5a13b88"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "3611751a0721d15b30859d93ad040d12"
server_key_iv: "5636610ee966420a3ea017f56979316e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.773772 18374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:48:12.774528 18403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.774742 18374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:12.774845 18374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "667ec5db9c0a4b8eb830e195f5a13b88"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "3611751a0721d15b30859d93ad040d12"
server_key_iv: "5636610ee966420a3ea017f56979316e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.774921 18374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:12.810755 18374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:12.813251 18374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:12.813424 18374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:12.813719 18374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:12.814121 18374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:12.814160 18374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.814193 18374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:12.814211 18374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.820358 18374 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:40687
I20260811 18:48:12.820432 18615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:40687 every 8 connection(s)
I20260811 18:48:12.820719 18374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:12.829367 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18374
I20260811 18:48:12.829493 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:12.829703 10076 external_mini_cluster.cc:1428] Setting key 1c3b5f302d0bfb711aafb7b9872e2738
I20260811 18:48:12.831032 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:12.821487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:45407 (local address 127.9.215.62:36683)
0811 18:48:12.821550 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:12.821552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:12.821788 (+ 236us) server_negotiation.cc:407] Connection header received
0811 18:48:12.822309 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:12.822310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:12.822323 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:12.822389 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:12.823586 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.824113 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.825243 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.825464 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.825527 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:12.827863 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:12.827865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:12.827865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:12.827873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:12.829253 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:12.829848 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:12.829850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:12.829850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:12.829923 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:12.830219 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:12.830220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:12.830221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:12.830395 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:12.830468 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:12.830852 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 18:48:12.830962 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:48:12.831260 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:12.821474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36683 (local address 127.9.215.1:45407)
0811 18:48:12.821634 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:12.821682 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:12.822188 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:12.822451 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:12.822454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:12.822742 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:12.823438 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:12.823446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.824248 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.824249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:12.825137 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:12.825141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.825559 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.825560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:12.825689 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:12.826239 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:12.826248 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:12.827764 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:12.829381 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:12.829391 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:12.829394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:12.829727 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:12.830002 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:12.830003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:12.830004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:12.830128 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:12.830463 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:12.830475 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:12.830672 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 18:48:12.830813 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 18:48:12.830999 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260811 18:48:12.831840 18617 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36683
I20260811 18:48:12.831947 18617 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:12.832123 18617 heartbeater.cc:511] Master 127.9.215.62:36683 requested a full tablet report, sending...
I20260811 18:48:12.832664 18266 ts_manager.cc:194] Registered new tserver with Master: 667ec5db9c0a4b8eb830e195f5a13b88 (127.9.215.1:40687)
I20260811 18:48:12.833297 18266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:45407
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:12.858722 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36683
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:12.945132 18632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:12.945358 18632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:12.945396 18632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:12.946702 18632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:12.946756 18632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:12.946786 18632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:12.948514 18632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:12.948803 18632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:12.949049 18632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:12.954432 18637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:12.954424 18643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:12.954424 18640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:12.954666 18632 server_base.cc:1034] running on GCE node
I20260811 18:48:12.954902 18632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:12.955188 18632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:12.956337 18632 hybrid_clock.cc:648] HybridClock initialized: now 1786474092956320 us; error 38 us; skew 500 ppm
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:12.959044 18632 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:12.959407 18632 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:12.959479 18632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:12.959611 18632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:12.960709 18632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "c03855f1ce0b4de5b3648e54e3344ff0"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "0ecea21e385cb9945ab5182f5346caee"
server_key_iv: "eb0da2d0323b93fa6939398eba47614e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.961012 18632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "c03855f1ce0b4de5b3648e54e3344ff0"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "0ecea21e385cb9945ab5182f5346caee"
server_key_iv: "eb0da2d0323b93fa6939398eba47614e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.962281 18632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:12.962998 18656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.963218 18632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:12.963312 18632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "c03855f1ce0b4de5b3648e54e3344ff0"
format_stamp: "Formatted at 2026-08-11 18:48:12 on dist-test-slave-pww0"
server_key: "0ecea21e385cb9945ab5182f5346caee"
server_key_iv: "eb0da2d0323b93fa6939398eba47614e"
server_key_version: "encryptionkey@0"
I20260811 18:48:12.963375 18632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:12.973395 18632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:12.975760 18632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:12.975883 18632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:12.976085 18632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:12.976378 18632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:12.976409 18632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.976431 18632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:12.976442 18632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:12.982228 18632 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:40475
I20260811 18:48:12.982290 18882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:40475 every 8 connection(s)
I20260811 18:48:12.982554 18632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:12.984822 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18632
I20260811 18:48:12.984932 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:12.985138 10076 external_mini_cluster.cc:1428] Setting key 24e48834127693be709f3205796ce0c4
Aug 11 18:48:12 dist-test-slave-pww0 krb5kdc[18194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474092, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:12.993633 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:12.983274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:39031 (local address 127.9.215.62:36683)
0811 18:48:12.983324 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:12.983342 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:12.983614 (+ 272us) server_negotiation.cc:407] Connection header received
0811 18:48:12.984189 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:12.984190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:12.984204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:12.984260 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:12.985543 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.986203 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.987701 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.988017 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.988143 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:12.990673 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:12.990675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:12.990675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:12.990684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:12.992150 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:12.992627 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:12.992629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:12.992629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:12.992689 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:12.992932 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:12.992933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:12.992933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:12.993080 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:12.993139 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:12.993435 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:48:12.993535 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:12.993846 18891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:12.983264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36683 (local address 127.9.215.2:39031)
0811 18:48:12.983420 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:12.983522 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:12.984078 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:12.984340 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:12.984344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:12.984699 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:12.985432 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:12.985441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.986323 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.986324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:12.987565 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:12.987570 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:12.988115 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:12.988117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:12.988284 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:12.988921 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:12.988933 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:12.990535 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:12.992247 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:12.992259 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:12.992263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:12.992519 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:12.992747 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:12.992748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:12.992748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:12.992875 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:12.993141 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:12.993150 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:12.993309 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 18:48:12.993407 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 18:48:12.993605 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260811 18:48:12.994403 18884 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36683
I20260811 18:48:12.994532 18884 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:12.994725 18884 heartbeater.cc:511] Master 127.9.215.62:36683 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
I20260811 18:48:12.995290 18266 ts_manager.cc:194] Registered new tserver with Master: c03855f1ce0b4de5b3648e54e3344ff0 (127.9.215.2:40475)
I20260811 18:48:12.995740 18266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:39031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:13.014840 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36683
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:13.101482 18895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:13.101734 18895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:13.101761 18895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:13.103025 18895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:13.103083 18895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:13.103113 18895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:13.104790 18895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:13.105055 18895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:13.105315 18895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:13.110814 18905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.110780 18904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:13.110970 18895 server_base.cc:1034] running on GCE node
W20260811 18:48:13.110809 18912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:13.111300 18895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:13.111604 18895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:13.112759 18895 hybrid_clock.cc:648] HybridClock initialized: now 1786474093112743 us; error 31 us; skew 500 ppm
Aug 11 18:48:13 dist-test-slave-pww0 krb5kdc[18194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474093, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:13.115602 18895 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:13.115964 18895 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:13.116020 18895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:13.116169 18895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:13.117295 18895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "b9a9f688e7f644148c26a7540bb80882"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "39ff32e009afb3484a30cdfa0bab624f"
server_key_iv: "9076b4fb3e187cbe9f893b7d2d18d6bf"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.117650 18895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "b9a9f688e7f644148c26a7540bb80882"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "39ff32e009afb3484a30cdfa0bab624f"
server_key_iv: "9076b4fb3e187cbe9f893b7d2d18d6bf"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.118978 18895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:13.119725 18926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.119884 18895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:13.119974 18895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "b9a9f688e7f644148c26a7540bb80882"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "39ff32e009afb3484a30cdfa0bab624f"
server_key_iv: "9076b4fb3e187cbe9f893b7d2d18d6bf"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.120038 18895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:13.133102 18895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:13.136018 18895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:13.136143 18895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:13.136351 18895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:13.136631 18895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:13.136660 18895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.136695 18895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:13.136708 18895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.142442 18895 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:37929
I20260811 18:48:13.142481 19144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:37929 every 8 connection(s)
I20260811 18:48:13.142760 18895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:48:13 dist-test-slave-pww0 krb5kdc[18194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474093, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:13.151160 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18895
I20260811 18:48:13.151270 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:13.151430 10076 external_mini_cluster.cc:1428] Setting key 13d518ca23859962601ae7d021814865
I20260811 18:48:13.152808 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.143528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:45921 (local address 127.9.215.62:36683)
0811 18:48:13.143584 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.143586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.143799 (+ 213us) server_negotiation.cc:407] Connection header received
0811 18:48:13.144314 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.144315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.144326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.144364 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.145403 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.145948 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.147101 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.147311 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.147377 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.149678 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.149680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.149680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:13.149688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.151157 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:13.151805 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:13.151806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:13.151807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:13.151872 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:13.152134 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:13.152135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:13.152136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:13.152265 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.152321 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.152616 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.152716 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:48:13.152995 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.143517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36683 (local address 127.9.215.3:45921)
0811 18:48:13.143665 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.143704 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.144199 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.144433 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.144436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.144712 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.145288 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.145294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.146083 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.146084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.146993 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.146996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.147418 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.147419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.147592 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.148110 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:13.148119 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.149568 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.151287 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:13.151299 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:13.151301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:13.151722 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:13.151931 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:13.151932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:13.151932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:13.152064 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:13.152321 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.152330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.152479 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.152570 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.152761 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260811 18:48:13.153446 19146 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36683
I20260811 18:48:13.153543 19146 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:13.153712 19146 heartbeater.cc:511] Master 127.9.215.62:36683 requested a full tablet report, sending...
I20260811 18:48:13.154071 18266 ts_manager.cc:194] Registered new tserver with Master: b9a9f688e7f644148c26a7540bb80882 (127.9.215.3:37929)
I20260811 18:48:13.154466 18266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:45921
I20260811 18:48:13.163256 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:48:13.171154 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.164790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58698 (local address 127.9.215.62:36683)
0811 18:48:13.164863 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.164865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.164972 (+ 107us) server_negotiation.cc:407] Connection header received
0811 18:48:13.165058 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.165059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.165073 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.165129 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.165778 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.166318 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.167072 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.167285 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.167361 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.168665 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.168666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.168667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:13.168676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.169939 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:13.170317 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:13.170319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:13.170320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:13.170382 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:13.170550 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:13.170551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:13.170552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:13.170732 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.170832 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.170964 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.171054 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:48:13.172956 18266 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:13.174731 18266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:13.182260 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.178011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.9.215.1:40687)
0811 18:48:13.178310 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.178315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.178552 (+ 237us) server_negotiation.cc:407] Connection header received
0811 18:48:13.178603 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.178606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.178621 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.178689 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:13.178889 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.180110 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.181318 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.181835 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.181905 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.181970 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.182117 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.182162 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
I20260811 18:48:13.182286 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.177858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:40687 (local address 127.0.0.1:35908)
0811 18:48:13.178412 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.178423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.178472 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.178781 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.178782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.178784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:13.178820 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.178821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.180220 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.180221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.181239 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.181242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.181924 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.181925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.182014 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.182026 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.182091 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.182155 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":25,"thread_start_us":106,"threads_started":1}
I20260811 18:48:13.182471 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.178077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.9.215.3:37929)
0811 18:48:13.178399 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.178405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.178416 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:13.178584 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.178587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.178604 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.178690 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:13.178872 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.179674 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.181068 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.181837 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.181908 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.181970 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.182338 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.182395 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":100,"threads_started":1}
I20260811 18:48:13.182430 19166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.178066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:40475 (local address 127.0.0.1:50046)
0811 18:48:13.178543 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.178553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.178599 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.178851 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.178852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.178854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:13.178912 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.178913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.180285 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.180286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.181270 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.181273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.182226 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.182226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.182290 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.182293 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.182338 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.182376 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260811 18:48:13.182766 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.178176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50046 (local address 127.9.215.2:40475)
0811 18:48:13.178495 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.178499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.178651 (+ 152us) server_negotiation.cc:407] Connection header received
0811 18:48:13.178691 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.178693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.178707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.178770 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:13.178991 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.180102 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.181370 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.182077 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.182139 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.182192 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.182512 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.182683 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":84,"threads_started":1}
I20260811 18:48:13.182294 19164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.177849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:37929 (local address 127.0.0.1:48000)
0811 18:48:13.178232 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.178343 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.178453 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.178702 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.178706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.178708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:13.178793 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.178796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.179773 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.179774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.180968 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.180971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.182007 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.182008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.182073 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.182078 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.182134 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.182184 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":68,"threads_started":1}
I20260811 18:48:13.183537 18490 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8e931e3f834374a2ec51a7b9a3059d (DEFAULT_TABLE table=test-table [id=2a801e9c8826471ca4b5baac14451441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:13.183686 19013 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8e931e3f834374a2ec51a7b9a3059d (DEFAULT_TABLE table=test-table [id=2a801e9c8826471ca4b5baac14451441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:13.183691 18754 tablet_service.cc:1467] Processing CreateTablet for tablet 2d8e931e3f834374a2ec51a7b9a3059d (DEFAULT_TABLE table=test-table [id=2a801e9c8826471ca4b5baac14451441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:13.183971 18490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8e931e3f834374a2ec51a7b9a3059d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.183990 18754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8e931e3f834374a2ec51a7b9a3059d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.184036 19013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d8e931e3f834374a2ec51a7b9a3059d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.186988 19173 tablet_bootstrap.cc:492] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Bootstrap starting.
I20260811 18:48:13.187569 19174 tablet_bootstrap.cc:492] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Bootstrap starting.
I20260811 18:48:13.187568 19176 tablet_bootstrap.cc:492] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88: Bootstrap starting.
I20260811 18:48:13.187935 19173 tablet_bootstrap.cc:654] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.188228 19176 tablet_bootstrap.cc:654] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.188227 19174 tablet_bootstrap.cc:654] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.188557 19173 log.cc:824] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.188850 19174 log.cc:824] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.188850 19176 log.cc:824] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.189380 19173 tablet_bootstrap.cc:492] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: No bootstrap required, opened a new log
I20260811 18:48:13.189465 19173 ts_tablet_manager.cc:1397] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:48:13.189603 19176 tablet_bootstrap.cc:492] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88: No bootstrap required, opened a new log
I20260811 18:48:13.189677 19176 ts_tablet_manager.cc:1397] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:48:13.189978 19174 tablet_bootstrap.cc:492] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: No bootstrap required, opened a new log
I20260811 18:48:13.190088 19174 ts_tablet_manager.cc:1397] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:48:13.191057 19173 raft_consensus.cc:348] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.191213 19173 raft_consensus.cc:374] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.191236 19173 raft_consensus.cc:729] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a9f688e7f644148c26a7540bb80882, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.191339 19173 consensus_queue.cc:260] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.191645 19146 heartbeater.cc:503] Master 127.9.215.62:36683 was elected leader, sending a full tablet report...
I20260811 18:48:13.191728 19176 raft_consensus.cc:348] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.191903 19173 ts_tablet_manager.cc:1428] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:48:13.191939 19176 raft_consensus.cc:374] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.191990 19176 raft_consensus.cc:729] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667ec5db9c0a4b8eb830e195f5a13b88, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.192102 19176 consensus_queue.cc:260] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.192405 18617 heartbeater.cc:503] Master 127.9.215.62:36683 was elected leader, sending a full tablet report...
I20260811 18:48:13.192405 19176 ts_tablet_manager.cc:1428] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:48:13.192549 19174 raft_consensus.cc:348] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.192727 19174 raft_consensus.cc:374] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.192767 19174 raft_consensus.cc:729] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03855f1ce0b4de5b3648e54e3344ff0, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.192875 19174 consensus_queue.cc:260] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.193148 19174 ts_tablet_manager.cc:1428] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:48:13.193167 18884 heartbeater.cc:503] Master 127.9.215.62:36683 was elected leader, sending a full tablet report...
I20260811 18:48:13.207309 19184 raft_consensus.cc:482] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:13.207444 19184 raft_consensus.cc:504] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.207872 19184 leader_election.cc:288] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9a9f688e7f644148c26a7540bb80882 (127.9.215.3:37929), 667ec5db9c0a4b8eb830e195f5a13b88 (127.9.215.1:40687)
I20260811 18:48:13.211964 18891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.208147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:37929 (local address 127.9.215.2:39971)
0811 18:48:13.208193 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.208205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.208263 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.208414 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.208415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.208417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:13.208507 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.208509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.209458 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.209460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.210815 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.210821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.211624 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.211625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.211743 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.211745 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.211812 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.211859 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":26}
I20260811 18:48:13.211999 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.208249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:39971 (local address 127.9.215.3:37929)
0811 18:48:13.208301 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.208303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.208312 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:13.208337 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.208338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.208348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.208401 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:13.208587 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.209337 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.210913 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.211524 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.211588 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.211601 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.211874 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.211924 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 18:48:13.211999 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.208198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:41395 (local address 127.9.215.1:40687)
0811 18:48:13.208248 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.208250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.208457 (+ 207us) server_negotiation.cc:407] Connection header received
0811 18:48:13.208508 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.208509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.208520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.208563 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:13.208702 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.209394 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.210912 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.211512 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.211589 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.211601 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.211873 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.211924 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:48:13.211970 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.208147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:40687 (local address 127.9.215.2:41395)
0811 18:48:13.208400 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.208409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.208461 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.208589 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.208590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.208592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:13.208645 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.208646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.209476 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.209477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.210827 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:13.210828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.211624 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.211625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:13.211737 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.211742 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.211816 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.211859 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":87,"thread_start_us":129,"threads_started":1}
I20260811 18:48:13.212406 18528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e931e3f834374a2ec51a7b9a3059d" candidate_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" is_pre_election: true
I20260811 18:48:13.212404 19048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e931e3f834374a2ec51a7b9a3059d" candidate_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a9f688e7f644148c26a7540bb80882" is_pre_election: true
I20260811 18:48:13.212579 18528 raft_consensus.cc:2454] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c03855f1ce0b4de5b3648e54e3344ff0 in term 0.
I20260811 18:48:13.212579 19048 raft_consensus.cc:2454] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c03855f1ce0b4de5b3648e54e3344ff0 in term 0.
I20260811 18:48:13.212859 18660 leader_election.cc:302] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 667ec5db9c0a4b8eb830e195f5a13b88, c03855f1ce0b4de5b3648e54e3344ff0; no voters:
I20260811 18:48:13.213027 19184 raft_consensus.cc:2781] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:13.213105 19184 raft_consensus.cc:482] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:13.213133 19184 raft_consensus.cc:3037] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.213848 19184 raft_consensus.cc:504] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.213989 19184 leader_election.cc:288] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [CANDIDATE]: Term 1 election: Requested vote from peers b9a9f688e7f644148c26a7540bb80882 (127.9.215.3:37929), 667ec5db9c0a4b8eb830e195f5a13b88 (127.9.215.1:40687)
I20260811 18:48:13.214154 19048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e931e3f834374a2ec51a7b9a3059d" candidate_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9a9f688e7f644148c26a7540bb80882"
I20260811 18:48:13.214243 19048 raft_consensus.cc:3037] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.214285 18528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d8e931e3f834374a2ec51a7b9a3059d" candidate_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667ec5db9c0a4b8eb830e195f5a13b88"
I20260811 18:48:13.214360 18528 raft_consensus.cc:3037] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.215241 18528 raft_consensus.cc:2454] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c03855f1ce0b4de5b3648e54e3344ff0 in term 1.
I20260811 18:48:13.215241 19048 raft_consensus.cc:2454] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c03855f1ce0b4de5b3648e54e3344ff0 in term 1.
I20260811 18:48:13.215492 18660 leader_election.cc:302] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 667ec5db9c0a4b8eb830e195f5a13b88, c03855f1ce0b4de5b3648e54e3344ff0; no voters:
I20260811 18:48:13.215634 19184 raft_consensus.cc:2781] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:13.215832 19184 raft_consensus.cc:686] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 1 LEADER]: Becoming Leader. State: Replica: c03855f1ce0b4de5b3648e54e3344ff0, State: Running, Role: LEADER
I20260811 18:48:13.215950 19184 consensus_queue.cc:237] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } }
I20260811 18:48:13.216821 18266 catalog_manager.cc:5495] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c03855f1ce0b4de5b3648e54e3344ff0 (127.9.215.2). New cstate: current_term: 1 leader_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c03855f1ce0b4de5b3648e54e3344ff0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40475 } health_report { overall_health: HEALTHY } } }
I20260811 18:48:13.228991 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.226447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.9.215.2:40475)
0811 18:48:13.226512 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.226514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.226524 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:13.226564 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.226565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.226588 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.226691 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:13.226847 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.227332 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.228173 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:13.228380 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:13.228440 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:13.228608 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:13.228738 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:13.228815 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.228854 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.228879 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:48:13.231067 18528 raft_consensus.cc:1261] T 2d8e931e3f834374a2ec51a7b9a3059d P 667ec5db9c0a4b8eb830e195f5a13b88 [term 1 FOLLOWER]: Refusing update from remote peer c03855f1ce0b4de5b3648e54e3344ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:13.231420 19048 raft_consensus.cc:1261] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 1 FOLLOWER]: Refusing update from remote peer c03855f1ce0b4de5b3648e54e3344ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:13.231516 19186 consensus_queue.cc:1035] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "667ec5db9c0a4b8eb830e195f5a13b88" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:13.231732 19186 consensus_queue.cc:1035] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9a9f688e7f644148c26a7540bb80882" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:48:13.233165 18890 tablet.cc:2369] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:48:13.234014 19196 mvcc.cc:205] Tried to move back new op lower bound from 7317397885871497216 to 7317397885813260288. Current Snapshot: MvccSnapshot[applied={T|T < 7317397885871497216}]
I20260811 18:48:13.237555 18265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:13.237686 18265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:13.238519 18265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55fc8d03d97a49198c570852f06dd71a: Sending DeleteTablet for 3 replicas of tablet 2d8e931e3f834374a2ec51a7b9a3059d
I20260811 18:48:13.238950 18490 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d8e931e3f834374a2ec51a7b9a3059d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:13 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:35908
I20260811 18:48:13.238950 19013 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d8e931e3f834374a2ec51a7b9a3059d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:13 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:48000
I20260811 18:48:13.239224 19223 tablet_replica.cc:330] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: stopping tablet replica
I20260811 18:48:13.239346 19223 raft_consensus.cc:2229] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:13.239336 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18374
I20260811 18:48:13.239382 18754 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d8e931e3f834374a2ec51a7b9a3059d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:13 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:50046
I20260811 18:48:13.242877 19228 tablet_replica.cc:330] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: stopping tablet replica
I20260811 18:48:13.243079 19228 raft_consensus.cc:2229] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:13.243309 19228 raft_consensus.cc:2258] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:13.254969 19228 ts_tablet_manager.cc:1905] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:13.257305 19228 ts_tablet_manager.cc:1918] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:13.257609 19228 log.cc:1196] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/2d8e931e3f834374a2ec51a7b9a3059d
I20260811 18:48:13.258073 19228 ts_tablet_manager.cc:1939] T 2d8e931e3f834374a2ec51a7b9a3059d P c03855f1ce0b4de5b3648e54e3344ff0: Deleting consensus metadata
I20260811 18:48:13.258854 18245 catalog_manager.cc:4841] TS c03855f1ce0b4de5b3648e54e3344ff0 (127.9.215.2:40475): tablet 2d8e931e3f834374a2ec51a7b9a3059d (table test-table [id=2a801e9c8826471ca4b5baac14451441]) successfully deleted
I20260811 18:48:13.269255 19223 raft_consensus.cc:2258] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:48:13.273379 18245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:40687 (error 108)
I20260811 18:48:13.273808 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18632
I20260811 18:48:13.273959 19223 ts_tablet_manager.cc:1905] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:13.274148 19164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.274010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:40687 (local address 127.0.0.1:35910)
0811 18:48:13.274069 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.274109 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 18:48:13.274310 18245 catalog_manager.cc:4584] TS 667ec5db9c0a4b8eb830e195f5a13b88 (127.9.215.1:40687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d8e931e3f834374a2ec51a7b9a3059d: Network error: Client connection negotiation failed: client connection to 127.9.215.1:40687: connect: Connection refused (error 111)
I20260811 18:48:13.287524 19223 ts_tablet_manager.cc:1918] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:13.287746 19223 log.cc:1196] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/2d8e931e3f834374a2ec51a7b9a3059d
I20260811 18:48:13.288129 19223 ts_tablet_manager.cc:1939] T 2d8e931e3f834374a2ec51a7b9a3059d P b9a9f688e7f644148c26a7540bb80882: Deleting consensus metadata
I20260811 18:48:13.288791 18239 catalog_manager.cc:4841] TS b9a9f688e7f644148c26a7540bb80882 (127.9.215.3:37929): tablet 2d8e931e3f834374a2ec51a7b9a3059d (table test-table [id=2a801e9c8826471ca4b5baac14451441]) successfully deleted
I20260811 18:48:13.294441 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18895
I20260811 18:48:13.313832 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18215
2026-08-11T18:48:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2273 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T18:48:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:13Z Disabled control of system clock
I20260811 18:48:13.358443 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:38747
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:38747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:48:13.448804 19240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:13.449039 19240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:13.449062 19240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:13.450249 19240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:13.450312 19240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:13.450323 19240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:13.450335 19240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:13.450345 19240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:13.451745 19240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:38747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:38747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:13.451972 19240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:13.452243 19240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:13.457221 19240 server_base.cc:1034] running on GCE node
W20260811 18:48:13.457146 19247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.457142 19250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.457146 19255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:13.457681 19240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:13.458000 19240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:13.459144 19240 hybrid_clock.cc:648] HybridClock initialized: now 1786474093459126 us; error 35 us; skew 500 ppm
I20260811 18:48:13.460500 19240 webserver.cc:460] Webserver started at http://127.9.215.62:45753/ using document root <none> and password file <none>
I20260811 18:48:13.460736 19240 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:13.460812 19240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:13.460939 19240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:13.462066 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "7b6b332f43b5456081550fdbdf1b2742"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "07a1d646264b155a7da56aa0249d6e8b"
server_key_iv: "4d4bd1dc47a4799ef8b8be193ae7971e"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.462417 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "7b6b332f43b5456081550fdbdf1b2742"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "07a1d646264b155a7da56aa0249d6e8b"
server_key_iv: "4d4bd1dc47a4799ef8b8be193ae7971e"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.463845 19240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:13.464619 19267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.464776 19240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:13.464866 19240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "7b6b332f43b5456081550fdbdf1b2742"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "07a1d646264b155a7da56aa0249d6e8b"
server_key_iv: "4d4bd1dc47a4799ef8b8be193ae7971e"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.464926 19240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:13.468421 19240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:13.468730 19240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:13.468864 19240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:13.473287 19240 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:38747
I20260811 18:48:13.473326 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:38747 every 8 connection(s)
I20260811 18:48:13.473702 19240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:13.473871 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19240
I20260811 18:48:13.473956 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:13.474151 10076 external_mini_cluster.cc:1428] Setting key 2d8bfc6c0c613f70578f408a0eb744a1
I20260811 18:48:13.474537 19372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.477309 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.474700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.9.215.62:38747)
0811 18:48:13.475223 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.475233 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.475295 (+ 62us) server_negotiation.cc:407] Connection header received
0811 18:48:13.475917 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.475925 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.475981 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.476118 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.476869 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.476873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.476874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.476878 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.476922 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.476933 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.476992 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.477055 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.477117 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":382,"threads_started":1}
I20260811 18:48:13.477553 19372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742: Bootstrap starting.
I20260811 18:48:13.478157 19372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.478431 19372 log.cc:824] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.479210 19372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742: No bootstrap required, opened a new log
I20260811 18:48:13.480674 19372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6b332f43b5456081550fdbdf1b2742" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38747 } }
I20260811 18:48:13.480834 19372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.480868 19372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6b332f43b5456081550fdbdf1b2742, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.480994 19372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6b332f43b5456081550fdbdf1b2742" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38747 } }
I20260811 18:48:13.481072 19372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:13.481104 19372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:13.481154 19372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.481737 19372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6b332f43b5456081550fdbdf1b2742" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38747 } }
I20260811 18:48:13.481845 19372 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6b332f43b5456081550fdbdf1b2742; no voters:
I20260811 18:48:13.482017 19372 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:13.482272 19381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:13.482327 19372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:13.482419 19381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [term 1 LEADER]: Becoming Leader. State: Replica: 7b6b332f43b5456081550fdbdf1b2742, State: Running, Role: LEADER
I20260811 18:48:13.482542 19381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6b332f43b5456081550fdbdf1b2742" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38747 } }
I20260811 18:48:13.482937 19382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b6b332f43b5456081550fdbdf1b2742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6b332f43b5456081550fdbdf1b2742" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38747 } } }
I20260811 18:48:13.482946 19386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6b332f43b5456081550fdbdf1b2742. Latest consensus state: current_term: 1 leader_uuid: "7b6b332f43b5456081550fdbdf1b2742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6b332f43b5456081550fdbdf1b2742" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38747 } } }
I20260811 18:48:13.483069 19382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:13.483078 19386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:13.483800 19399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:13.484356 19399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:13.485937 19399 catalog_manager.cc:1347] Generated new cluster ID: 3a1a1f05895b4e1db53fe5a092b3e80e
I20260811 18:48:13.485992 19399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:13.506830 19399 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:13.507025 19399 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:13.519577 19399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742: Generated new TSK 0
I20260811 18:48:13.519824 19399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:13.521219 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38747
--builtin_ntp_servers=127.9.215.20:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:48:13.606298 19420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:13.606526 19420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:13.606560 19420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:13.607800 19420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:13.607844 19420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:13.607880 19420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:13.609491 19420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:13.609782 19420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:13.610107 19420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:13.615240 19435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.615279 19436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.615240 19438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:13.615690 19420 server_base.cc:1034] running on GCE node
I20260811 18:48:13.615958 19420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:13.616219 19420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:13.617363 19420 hybrid_clock.cc:648] HybridClock initialized: now 1786474093617351 us; error 31 us; skew 500 ppm
I20260811 18:48:13.618706 19420 webserver.cc:460] Webserver started at http://127.9.215.1:41093/ using document root <none> and password file <none>
I20260811 18:48:13.618975 19420 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:13.619026 19420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:13.619158 19420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:13.620357 19420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "19a7a028a7444cbaa2c6ff00f82f3354"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "c52612ed46692fecdae4bf03382e6552"
server_key_iv: "f9b0eab8ee48343c65284a5c4772385a"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.620689 19420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "19a7a028a7444cbaa2c6ff00f82f3354"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "c52612ed46692fecdae4bf03382e6552"
server_key_iv: "f9b0eab8ee48343c65284a5c4772385a"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.621953 19420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:13.622669 19449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.622922 19420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:13.623013 19420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "19a7a028a7444cbaa2c6ff00f82f3354"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "c52612ed46692fecdae4bf03382e6552"
server_key_iv: "f9b0eab8ee48343c65284a5c4772385a"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.623075 19420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:13.626521 19420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:13.626812 19420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:13.626933 19420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:13.627115 19420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:13.627410 19420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:13.627445 19420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.627489 19420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:13.627508 19420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.633067 19420 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:43569
I20260811 18:48:13.633142 19660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:43569 every 8 connection(s)
I20260811 18:48:13.633486 19420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:13.636412 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.634197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:40651 (local address 127.9.215.62:38747)
0811 18:48:13.634290 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.634292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.634483 (+ 191us) server_negotiation.cc:407] Connection header received
0811 18:48:13.634964 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.634965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.634970 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.635020 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.636227 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.636228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.636229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.636232 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.636250 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.636253 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.636289 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.636344 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.636355 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:48:13.636620 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.634177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:38747 (local address 127.9.215.1:40651)
0811 18:48:13.634360 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.634396 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.634885 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.635082 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.635085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.635093 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.636076 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.636077 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.636108 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.636139 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.636293 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.636295 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.636297 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.636349 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.636403 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260811 18:48:13.637028 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19420
I20260811 18:48:13.637131 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:13.637145 19662 heartbeater.cc:347] Connected to a master server at 127.9.215.62:38747
I20260811 18:48:13.637245 19662 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:13.637337 10076 external_mini_cluster.cc:1428] Setting key ef0c38c76c4305c6f0ce952912044f78
I20260811 18:48:13.637409 19662 heartbeater.cc:511] Master 127.9.215.62:38747 requested a full tablet report, sending...
I20260811 18:48:13.638038 19293 ts_manager.cc:194] Registered new tserver with Master: 19a7a028a7444cbaa2c6ff00f82f3354 (127.9.215.1:43569)
I20260811 18:48:13.638304 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38747
--builtin_ntp_servers=127.9.215.20:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:48:13.723277 19665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:13.723569 19665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:13.723613 19665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:13.724915 19665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:13.724963 19665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:13.725001 19665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:13.726550 19665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:13.726857 19665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:13.727165 19665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:13.732203 19677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.732218 19685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.732323 19681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:13.732769 19665 server_base.cc:1034] running on GCE node
I20260811 18:48:13.733019 19665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:13.733290 19665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:13.734467 19665 hybrid_clock.cc:648] HybridClock initialized: now 1786474093734445 us; error 36 us; skew 500 ppm
I20260811 18:48:13.735935 19665 webserver.cc:460] Webserver started at http://127.9.215.2:40187/ using document root <none> and password file <none>
I20260811 18:48:13.736164 19665 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:13.736215 19665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:13.736341 19665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:13.737434 19665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "b73bd424954e47d381a8f8e371abcb8c"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "0ed6b5e1dbaf04f710290d205060f477"
server_key_iv: "dd124db917db666b2547ce9766b947b8"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.737779 19665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "b73bd424954e47d381a8f8e371abcb8c"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "0ed6b5e1dbaf04f710290d205060f477"
server_key_iv: "dd124db917db666b2547ce9766b947b8"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.739035 19665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:13.739777 19695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.739916 19665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:13.740006 19665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "b73bd424954e47d381a8f8e371abcb8c"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "0ed6b5e1dbaf04f710290d205060f477"
server_key_iv: "dd124db917db666b2547ce9766b947b8"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.740067 19665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:13.743512 19665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:13.743834 19665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:13.743964 19665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:13.744174 19665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:13.744480 19665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:13.744514 19665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.744542 19665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:13.744565 19665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.750315 19665 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:40725
I20260811 18:48:13.750371 19914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:40725 every 8 connection(s)
I20260811 18:48:13.750730 19665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:13.753737 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.751484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:59379 (local address 127.9.215.62:38747)
0811 18:48:13.751547 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.751549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.751799 (+ 250us) server_negotiation.cc:407] Connection header received
0811 18:48:13.752278 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.752279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.752286 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.752336 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.753492 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.753493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.753493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.753498 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.753514 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.753518 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.753572 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.753660 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.753673 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:48:13.753924 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.751432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:38747 (local address 127.9.215.2:59379)
0811 18:48:13.751671 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.751709 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.752187 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.752404 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.752408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.752417 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.753327 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.753328 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.753361 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.753392 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.753577 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.753580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.753582 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.753666 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.753722 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":118,"threads_started":1}
I20260811 18:48:13.754113 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19665
I20260811 18:48:13.754211 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:13.754415 10076 external_mini_cluster.cc:1428] Setting key 24fc9fcbf1852edd3a03270a7a4ade5d
I20260811 18:48:13.754403 19916 heartbeater.cc:347] Connected to a master server at 127.9.215.62:38747
I20260811 18:48:13.754498 19916 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:13.754674 19916 heartbeater.cc:511] Master 127.9.215.62:38747 requested a full tablet report, sending...
I20260811 18:48:13.755060 19294 ts_manager.cc:194] Registered new tserver with Master: b73bd424954e47d381a8f8e371abcb8c (127.9.215.2:40725)
I20260811 18:48:13.755517 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38747
--builtin_ntp_servers=127.9.215.20:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:48:13.839653 19920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:13.839896 19920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:13.839929 19920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:13.841184 19920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:13.841230 19920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:13.841277 19920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:13.842797 19920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:13.843071 19920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:13.843382 19920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:13.848652 19920 server_base.cc:1034] running on GCE node
W20260811 18:48:13.848614 19932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.848618 19930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:13.848608 19928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:13.849120 19920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:13.849380 19920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:13.850543 19920 hybrid_clock.cc:648] HybridClock initialized: now 1786474093850525 us; error 29 us; skew 500 ppm
I20260811 18:48:13.851881 19920 webserver.cc:460] Webserver started at http://127.9.215.3:34147/ using document root <none> and password file <none>
I20260811 18:48:13.852116 19920 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:13.852185 19920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:13.852316 19920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:13.853649 19920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "268488c7c5594930bc3ce9e8202fedc8"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "8644fe1b66fee144c2e66b908ef2c433"
server_key_iv: "fdba282fc8b4675141900437c05dfe96"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.854094 19920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "268488c7c5594930bc3ce9e8202fedc8"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "8644fe1b66fee144c2e66b908ef2c433"
server_key_iv: "fdba282fc8b4675141900437c05dfe96"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.855511 19920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:13.856372 19943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.856550 19920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:13.856667 19920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "268488c7c5594930bc3ce9e8202fedc8"
format_stamp: "Formatted at 2026-08-11 18:48:13 on dist-test-slave-pww0"
server_key: "8644fe1b66fee144c2e66b908ef2c433"
server_key_iv: "fdba282fc8b4675141900437c05dfe96"
server_key_version: "encryptionkey@0"
I20260811 18:48:13.856742 19920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:13.860229 19920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:13.860517 19920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:13.860646 19920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:13.860834 19920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:13.861167 19920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:13.861212 19920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.861241 19920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:13.861265 19920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:13.866807 19920 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:36675
I20260811 18:48:13.866853 20162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:36675 every 8 connection(s)
I20260811 18:48:13.867213 19920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:13.870199 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.868007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:45077 (local address 127.9.215.62:38747)
0811 18:48:13.868055 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.868057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.868295 (+ 238us) server_negotiation.cc:407] Connection header received
0811 18:48:13.868853 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.868854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.868860 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.868897 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.870001 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.870002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.870003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.870007 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.870021 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.870026 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.870081 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.870129 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.870140 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:48:13.870390 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.867991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:38747 (local address 127.9.215.3:45077)
0811 18:48:13.868177 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.868212 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.868747 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.868972 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.868975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.868983 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.869859 (+ 876us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.869859 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.869890 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.869917 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.870079 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.870083 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.870085 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.870136 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.870187 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260811 18:48:13.870846 20164 heartbeater.cc:347] Connected to a master server at 127.9.215.62:38747
I20260811 18:48:13.870927 20164 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:13.871073 20164 heartbeater.cc:511] Master 127.9.215.62:38747 requested a full tablet report, sending...
I20260811 18:48:13.871102 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19920
I20260811 18:48:13.871196 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:13.871382 10076 external_mini_cluster.cc:1428] Setting key ac6ed4314cd4cb6ee8cc41baa4d8ee19
I20260811 18:48:13.871438 19294 ts_manager.cc:194] Registered new tserver with Master: 268488c7c5594930bc3ce9e8202fedc8 (127.9.215.3:36675)
I20260811 18:48:13.872186 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:48:13.874238 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.873396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.9.215.62:38747)
0811 18:48:13.873432 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.873434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.873527 (+ 93us) server_negotiation.cc:407] Connection header received
0811 18:48:13.873578 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.873578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.873583 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.873627 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.874088 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.874089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.874090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.874093 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.874105 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.874108 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.874147 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.874177 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.874188 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 18:48:13.875793 19294 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:13.877419 19294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:13.881242 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.879499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43569 (local address 127.0.0.1:58990)
0811 18:48:13.879937 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.879952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.880001 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.880278 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.880279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.880280 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.880812 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.880812 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.880830 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.880856 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.881072 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.881087 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.881090 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.881152 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.881166 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":105,"threads_started":1}
I20260811 18:48:13.881259 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.879778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58990 (local address 127.9.215.1:43569)
0811 18:48:13.879974 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.879978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.879991 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:48:13.880050 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.880053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.880063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.880157 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.880947 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.880955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.880955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.880959 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.880993 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.881018 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.881070 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.881168 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.881182 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260811 18:48:13.881368 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.879496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36675 (local address 127.0.0.1:58818)
0811 18:48:13.880078 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.880086 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.880133 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.880556 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.880557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.880560 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.881006 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.881007 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.881014 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.881026 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.881253 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.881254 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.881254 (+ 0us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.881293 (+ 39us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.881303 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":5,"thread_start_us":122,"threads_started":1}
I20260811 18:48:13.881503 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.879759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.9.215.2:40725)
0811 18:48:13.880102 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.880105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.880112 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:13.880137 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.880139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.880149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.880224 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.881092 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.881096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.881096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.881101 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.881134 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.881150 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.881206 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.881404 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.881421 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":82,"threads_started":1}
I20260811 18:48:13.881563 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.879690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.9.215.3:36675)
0811 18:48:13.879885 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.879890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.880149 (+ 259us) server_negotiation.cc:407] Connection header received
0811 18:48:13.880178 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.880181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.880190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.880257 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.881117 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.881126 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.881127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.881131 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.881160 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.881184 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.881231 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.881473 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.881487 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260811 18:48:13.881368 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.879496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:40725 (local address 127.0.0.1:54684)
0811 18:48:13.879902 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.879947 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.880002 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.880221 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.880227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.880233 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.880883 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.880883 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.880888 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.880896 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.881231 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.881232 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.881233 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.881276 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.881289 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":77,"threads_started":1}
I20260811 18:48:13.882222 19787 tablet_service.cc:1467] Processing CreateTablet for tablet 03dab77b9e3b4dbba07ebd20de61fd60 (DEFAULT_TABLE table=test-table [id=3d737322bb1f4c0795c348bd4c6c1214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:13.882346 20045 tablet_service.cc:1467] Processing CreateTablet for tablet 03dab77b9e3b4dbba07ebd20de61fd60 (DEFAULT_TABLE table=test-table [id=3d737322bb1f4c0795c348bd4c6c1214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:13.882521 19787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03dab77b9e3b4dbba07ebd20de61fd60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.882710 20045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03dab77b9e3b4dbba07ebd20de61fd60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.883034 19539 tablet_service.cc:1467] Processing CreateTablet for tablet 03dab77b9e3b4dbba07ebd20de61fd60 (DEFAULT_TABLE table=test-table [id=3d737322bb1f4c0795c348bd4c6c1214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:13.883383 19539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03dab77b9e3b4dbba07ebd20de61fd60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:13.885169 20200 tablet_bootstrap.cc:492] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Bootstrap starting.
I20260811 18:48:13.885735 20200 tablet_bootstrap.cc:654] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.885947 20200 log.cc:824] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.886129 20201 tablet_bootstrap.cc:492] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Bootstrap starting.
I20260811 18:48:13.886699 20201 tablet_bootstrap.cc:654] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.886746 20200 tablet_bootstrap.cc:492] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: No bootstrap required, opened a new log
I20260811 18:48:13.886801 20200 ts_tablet_manager.cc:1397] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:13.887015 20201 log.cc:824] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.887193 20205 tablet_bootstrap.cc:492] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354: Bootstrap starting.
I20260811 18:48:13.887845 20201 tablet_bootstrap.cc:492] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: No bootstrap required, opened a new log
I20260811 18:48:13.887921 20201 ts_tablet_manager.cc:1397] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:13.888000 20205 tablet_bootstrap.cc:654] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:13.888372 20205 log.cc:824] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:13.888944 20200 raft_consensus.cc:348] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.889093 20200 raft_consensus.cc:374] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.889133 20200 raft_consensus.cc:729] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b73bd424954e47d381a8f8e371abcb8c, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.889245 20200 consensus_queue.cc:260] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.889397 20205 tablet_bootstrap.cc:492] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354: No bootstrap required, opened a new log
I20260811 18:48:13.889463 20205 ts_tablet_manager.cc:1397] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:13.889524 20200 ts_tablet_manager.cc:1428] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:48:13.889648 19916 heartbeater.cc:503] Master 127.9.215.62:38747 was elected leader, sending a full tablet report...
I20260811 18:48:13.890473 20201 raft_consensus.cc:348] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.890659 20201 raft_consensus.cc:374] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.890694 20201 raft_consensus.cc:729] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 268488c7c5594930bc3ce9e8202fedc8, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.890934 20201 consensus_queue.cc:260] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.891227 20201 ts_tablet_manager.cc:1428] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 18:48:13.891268 20164 heartbeater.cc:503] Master 127.9.215.62:38747 was elected leader, sending a full tablet report...
I20260811 18:48:13.892048 20205 raft_consensus.cc:348] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.892198 20205 raft_consensus.cc:374] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:13.892220 20205 raft_consensus.cc:729] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a7a028a7444cbaa2c6ff00f82f3354, State: Initialized, Role: FOLLOWER
I20260811 18:48:13.892306 20205 consensus_queue.cc:260] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.892544 20205 ts_tablet_manager.cc:1428] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:48:13.892591 19662 heartbeater.cc:503] Master 127.9.215.62:38747 was elected leader, sending a full tablet report...
I20260811 18:48:13.930632 20216 raft_consensus.cc:482] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:13.930758 20216 raft_consensus.cc:504] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.931142 20216 leader_election.cc:288] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 268488c7c5594930bc3ce9e8202fedc8 (127.9.215.3:36675), 19a7a028a7444cbaa2c6ff00f82f3354 (127.9.215.1:43569)
I20260811 18:48:13.931845 20218 raft_consensus.cc:482] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:13.931970 20218 raft_consensus.cc:504] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.932523 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.931417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:59939 (local address 127.9.215.3:36675)
0811 18:48:13.931482 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.931484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.931622 (+ 138us) server_negotiation.cc:407] Connection header received
0811 18:48:13.931661 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.931662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.931668 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.931724 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.932234 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.932235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.932236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.932241 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.932261 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.932265 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.932323 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.932432 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.932449 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:48:13.932591 20218 leader_election.cc:288] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b73bd424954e47d381a8f8e371abcb8c (127.9.215.2:40725), 268488c7c5594930bc3ce9e8202fedc8 (127.9.215.3:36675)
I20260811 18:48:13.932818 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.931322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36675 (local address 127.9.215.2:59939)
0811 18:48:13.931537 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.931546 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.931622 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.931741 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.931742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.931744 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.932124 (+ 380us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.932124 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.932134 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.932149 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.932327 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.932329 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.932330 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.932730 (+ 400us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.932743 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":121,"threads_started":1}
I20260811 18:48:13.932900 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.931312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43569 (local address 127.9.215.2:44909)
0811 18:48:13.931378 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.931391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.931428 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.931575 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.931576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.931578 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.932327 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.932328 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.932336 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.932345 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.932799 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.932799 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.932800 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.932843 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.932855 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:48:13.933039 20198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.931393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:44909 (local address 127.9.215.1:43569)
0811 18:48:13.931436 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.931438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.931449 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:13.931506 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.931507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.931513 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.931559 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.932426 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.932427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.932428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.932432 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.932449 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.932453 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.932499 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.932968 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.932981 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:13.933264 19574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "b73bd424954e47d381a8f8e371abcb8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" is_pre_election: true
I20260811 18:48:13.933382 19574 raft_consensus.cc:2454] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b73bd424954e47d381a8f8e371abcb8c in term 0.
I20260811 18:48:13.933398 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.932447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36675 (local address 127.9.215.1:50583)
0811 18:48:13.932496 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.932506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.932535 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.932708 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.932709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.932711 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.933105 (+ 394us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.933106 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.933114 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.933126 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.933291 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.933292 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.933293 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.933334 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.933345 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:48:13.933525 20083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "b73bd424954e47d381a8f8e371abcb8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "268488c7c5594930bc3ce9e8202fedc8" is_pre_election: true
I20260811 18:48:13.933635 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.932585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:50583 (local address 127.9.215.3:36675)
0811 18:48:13.932602 (+ 17us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.932604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.932611 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:13.932630 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.932630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.932634 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.932695 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.933195 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.933197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.933197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.933201 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.933214 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.933219 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.933278 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.933562 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.933576 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4}
I20260811 18:48:13.933607 19702 leader_election.cc:302] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19a7a028a7444cbaa2c6ff00f82f3354, b73bd424954e47d381a8f8e371abcb8c; no voters:
I20260811 18:48:13.933665 20083 raft_consensus.cc:2454] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b73bd424954e47d381a8f8e371abcb8c in term 0.
I20260811 18:48:13.933729 20216 raft_consensus.cc:2781] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:13.933775 20216 raft_consensus.cc:482] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:13.933795 20216 raft_consensus.cc:3037] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.934022 20083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "268488c7c5594930bc3ce9e8202fedc8" is_pre_election: true
I20260811 18:48:13.934059 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.932457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:40725 (local address 127.9.215.1:51541)
0811 18:48:13.932862 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:13.932870 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:13.932915 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:13.933049 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:13.933050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:13.933052 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:13.933742 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:13.933742 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:13.933749 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:13.933761 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:13.933937 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:13.933938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:13.933938 (+ 0us) client_negotiation.cc:769] Sending connection context
0811 18:48:13.933983 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:48:13.933995 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":107,"threads_started":1}
I20260811 18:48:13.934084 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.932651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:51541 (local address 127.9.215.2:40725)
0811 18:48:13.932694 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.932695 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.932918 (+ 223us) server_negotiation.cc:407] Connection header received
0811 18:48:13.932956 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.932957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.932961 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.933002 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.933847 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.933848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.933849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.933853 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.933864 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.933868 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.933897 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.934001 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.934022 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:48:13.934103 20083 raft_consensus.cc:2454] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19a7a028a7444cbaa2c6ff00f82f3354 in term 0.
I20260811 18:48:13.934233 19455 leader_election.cc:302] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19a7a028a7444cbaa2c6ff00f82f3354, 268488c7c5594930bc3ce9e8202fedc8; no voters:
I20260811 18:48:13.934271 19823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b73bd424954e47d381a8f8e371abcb8c" is_pre_election: true
I20260811 18:48:13.934342 20218 raft_consensus.cc:2781] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:13.934379 20218 raft_consensus.cc:482] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:13.934398 20218 raft_consensus.cc:3037] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.934768 20216 raft_consensus.cc:504] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.934923 20216 leader_election.cc:288] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [CANDIDATE]: Term 1 election: Requested vote from peers 268488c7c5594930bc3ce9e8202fedc8 (127.9.215.3:36675), 19a7a028a7444cbaa2c6ff00f82f3354 (127.9.215.1:43569)
I20260811 18:48:13.934959 19823 raft_consensus.cc:2379] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19a7a028a7444cbaa2c6ff00f82f3354 in current term 1: Already voted for candidate b73bd424954e47d381a8f8e371abcb8c in this term.
I20260811 18:48:13.935070 20083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "b73bd424954e47d381a8f8e371abcb8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "268488c7c5594930bc3ce9e8202fedc8"
I20260811 18:48:13.935104 19574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "b73bd424954e47d381a8f8e371abcb8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a7a028a7444cbaa2c6ff00f82f3354"
I20260811 18:48:13.935127 20083 raft_consensus.cc:3037] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:13.935228 20218 raft_consensus.cc:504] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.935359 19574 raft_consensus.cc:2379] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b73bd424954e47d381a8f8e371abcb8c in current term 1: Already voted for candidate 19a7a028a7444cbaa2c6ff00f82f3354 in this term.
I20260811 18:48:13.935442 20218 leader_election.cc:288] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [CANDIDATE]: Term 1 election: Requested vote from peers b73bd424954e47d381a8f8e371abcb8c (127.9.215.2:40725), 268488c7c5594930bc3ce9e8202fedc8 (127.9.215.3:36675)
I20260811 18:48:13.935636 20082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "268488c7c5594930bc3ce9e8202fedc8"
I20260811 18:48:13.936039 20083 raft_consensus.cc:2454] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b73bd424954e47d381a8f8e371abcb8c in term 1.
I20260811 18:48:13.936241 19697 leader_election.cc:302] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 268488c7c5594930bc3ce9e8202fedc8, b73bd424954e47d381a8f8e371abcb8c; no voters: 19a7a028a7444cbaa2c6ff00f82f3354
I20260811 18:48:13.936342 20216 raft_consensus.cc:2781] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:13.936359 19823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03dab77b9e3b4dbba07ebd20de61fd60" candidate_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b73bd424954e47d381a8f8e371abcb8c"
I20260811 18:48:13.936534 20216 raft_consensus.cc:686] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 1 LEADER]: Becoming Leader. State: Replica: b73bd424954e47d381a8f8e371abcb8c, State: Running, Role: LEADER
I20260811 18:48:13.936625 20216 consensus_queue.cc:237] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } }
I20260811 18:48:13.937162 19456 leader_election.cc:302] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19a7a028a7444cbaa2c6ff00f82f3354; no voters: 268488c7c5594930bc3ce9e8202fedc8, b73bd424954e47d381a8f8e371abcb8c
I20260811 18:48:13.937265 20218 raft_consensus.cc:2726] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 18:48:13.937448 19293 catalog_manager.cc:5495] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c reported cstate change: term changed from 0 to 1, leader changed from <none> to b73bd424954e47d381a8f8e371abcb8c (127.9.215.2). New cstate: current_term: 1 leader_uuid: "b73bd424954e47d381a8f8e371abcb8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b73bd424954e47d381a8f8e371abcb8c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 } health_report { overall_health: UNKNOWN } } }
I20260811 18:48:13.942896 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:13.942144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.9.215.2:40725)
0811 18:48:13.942202 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:13.942204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:13.942213 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:13.942246 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:13.942247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:13.942261 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:13.942307 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:13.942725 (+ 418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:13.942726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:13.942726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:13.942730 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:13.942748 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:13.942752 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:13.942791 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:13.942821 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 18:48:13.942832 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:48:13.944677 19574 raft_consensus.cc:1261] T 03dab77b9e3b4dbba07ebd20de61fd60 P 19a7a028a7444cbaa2c6ff00f82f3354 [term 1 FOLLOWER]: Refusing update from remote peer b73bd424954e47d381a8f8e371abcb8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:13.944722 20083 raft_consensus.cc:1261] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 1 FOLLOWER]: Refusing update from remote peer b73bd424954e47d381a8f8e371abcb8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:13.944905 20216 consensus_queue.cc:1035] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "19a7a028a7444cbaa2c6ff00f82f3354" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:13.945042 20222 consensus_queue.cc:1035] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "268488c7c5594930bc3ce9e8202fedc8" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:13.947543 20229 mvcc.cc:205] Tried to move back new op lower bound from 7317397888794755072 to 7317397888765054976. Current Snapshot: MvccSnapshot[applied={T|T < 7317397888794755072}]
I20260811 18:48:13.947705 20232 mvcc.cc:205] Tried to move back new op lower bound from 7317397888794755072 to 7317397888765054976. Current Snapshot: MvccSnapshot[applied={T|T < 7317397888794755072}]
I20260811 18:48:13.949488 19294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:13.949576 19294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:13.950297 19294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b6b332f43b5456081550fdbdf1b2742: Sending DeleteTablet for 3 replicas of tablet 03dab77b9e3b4dbba07ebd20de61fd60
I20260811 18:48:13.950609 19787 tablet_service.cc:1514] Processing DeleteTablet for tablet 03dab77b9e3b4dbba07ebd20de61fd60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:13 UTC) from {username='slave'} at 127.0.0.1:54684
I20260811 18:48:13.950691 19539 tablet_service.cc:1514] Processing DeleteTablet for tablet 03dab77b9e3b4dbba07ebd20de61fd60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:13 UTC) from {username='slave'} at 127.0.0.1:58990
I20260811 18:48:13.950838 20242 tablet_replica.cc:330] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: stopping tablet replica
I20260811 18:48:13.950834 20045 tablet_service.cc:1514] Processing DeleteTablet for tablet 03dab77b9e3b4dbba07ebd20de61fd60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:13 UTC) from {username='slave'} at 127.0.0.1:58818
I20260811 18:48:13.950947 20242 raft_consensus.cc:2229] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:13.951040 20242 raft_consensus.cc:2258] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:13.951268 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19420
I20260811 18:48:14.001348 20244 tablet_replica.cc:330] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: stopping tablet replica
I20260811 18:48:14.001556 20244 raft_consensus.cc:2229] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:14.001740 20244 raft_consensus.cc:2258] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:14.002146 20242 ts_tablet_manager.cc:1905] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:14.004410 20242 ts_tablet_manager.cc:1918] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:14.004519 20242 log.cc:1196] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/03dab77b9e3b4dbba07ebd20de61fd60
I20260811 18:48:14.004880 20242 ts_tablet_manager.cc:1939] T 03dab77b9e3b4dbba07ebd20de61fd60 P b73bd424954e47d381a8f8e371abcb8c: Deleting consensus metadata
I20260811 18:48:14.011448 20244 ts_tablet_manager.cc:1905] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:14.013631 20244 ts_tablet_manager.cc:1918] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:14.013743 20244 log.cc:1196] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/03dab77b9e3b4dbba07ebd20de61fd60
I20260811 18:48:14.014084 20244 ts_tablet_manager.cc:1939] T 03dab77b9e3b4dbba07ebd20de61fd60 P 268488c7c5594930bc3ce9e8202fedc8: Deleting consensus metadata
I20260811 18:48:14.018138 19271 catalog_manager.cc:4841] TS b73bd424954e47d381a8f8e371abcb8c (127.9.215.2:40725): tablet 03dab77b9e3b4dbba07ebd20de61fd60 (table test-table [id=3d737322bb1f4c0795c348bd4c6c1214]) successfully deleted
I20260811 18:48:14.018138 19270 catalog_manager.cc:4841] TS 268488c7c5594930bc3ce9e8202fedc8 (127.9.215.3:36675): tablet 03dab77b9e3b4dbba07ebd20de61fd60 (table test-table [id=3d737322bb1f4c0795c348bd4c6c1214]) successfully deleted
W20260811 18:48:14.025015 19273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:43569 (error 108)
I20260811 18:48:14.025332 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19665
I20260811 18:48:14.035748 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.035547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43569 (local address 127.0.0.1:59000)
0811 18:48:14.035614 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:14.035695 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:48:14.036031 19273 catalog_manager.cc:4584] TS 19a7a028a7444cbaa2c6ff00f82f3354 (127.9.215.1:43569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 03dab77b9e3b4dbba07ebd20de61fd60: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43569: connect: Connection refused (error 111)
I20260811 18:48:14.040074 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19920
I20260811 18:48:14.060992 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19240
2026-08-11T18:48:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (741 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 18:48:14.092728 10076 mini_oidc.cc:152] Starting JWKS server
I20260811 18:48:14.094007 10076 webserver.cc:460] Webserver started at https://127.0.0.1:40153/ using document root <none> and password file <none>
I20260811 18:48:14.094095 10076 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:48:14.094269 10076 webserver.cc:460] Webserver started at http://127.0.0.1:32859/ using document root <none> and password file <none>
2026-08-11T18:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:14Z Disabled control of system clock
I20260811 18:48:14.103608 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34543
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:39907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:34543
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40153/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/testchainca.cert with env {}
W20260811 18:48:14.189041 20255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:14.189281 20255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:14.189306 20255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:14.190600 20255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:14.190652 20255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:14.190665 20255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:14.190675 20255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:14.190686 20255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:48:14.190697 20255 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:48:14.190708 20255 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40153/jwks.json
I20260811 18:48:14.192554 20255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:34543
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40153/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:14.192838 20255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:14.193063 20255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:14.198343 20271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:14.198354 20263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.198362 20255 server_base.cc:1034] running on GCE node
W20260811 18:48:14.198333 20267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.198905 20255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:14.199235 20255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:14.200398 20255 hybrid_clock.cc:648] HybridClock initialized: now 1786474094200375 us; error 43 us; skew 500 ppm
I20260811 18:48:14.204754 20255 webserver.cc:460] Webserver started at http://127.9.215.62:38573/ using document root <none> and password file <none>
I20260811 18:48:14.205029 20255 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:14.205083 20255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:14.205217 20255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:14.206358 20255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "875a2ab3a78b47ff8ff36fbdb3a28734"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "2a503d40b40e7a944068183b87298a7d"
server_key_iv: "39ed19079f0d44800d6ca2502e473be2"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.206710 20255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "875a2ab3a78b47ff8ff36fbdb3a28734"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "2a503d40b40e7a944068183b87298a7d"
server_key_iv: "39ed19079f0d44800d6ca2502e473be2"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.208151 20255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:14.208940 20278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:14.209096 20255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:14.209177 20255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "875a2ab3a78b47ff8ff36fbdb3a28734"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "2a503d40b40e7a944068183b87298a7d"
server_key_iv: "39ed19079f0d44800d6ca2502e473be2"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.209242 20255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:14.220214 20255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:14.220584 20255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:14.220746 20255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:14.225102 20255 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:34543
I20260811 18:48:14.225138 20386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:34543 every 8 connection(s)
I20260811 18:48:14.225548 20255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:14.226644 20389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:14.228677 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20255
I20260811 18:48:14.228785 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:14.228950 10076 external_mini_cluster.cc:1428] Setting key 007a176a9e2450be6a423211ad03a057
I20260811 18:48:14.229208 20389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734: Bootstrap starting.
I20260811 18:48:14.230193 20389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:14.230584 20389 log.cc:824] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:14.231288 20389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734: No bootstrap required, opened a new log
I20260811 18:48:14.233181 20389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34543 } }
I20260811 18:48:14.233347 20389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:14.233383 20389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875a2ab3a78b47ff8ff36fbdb3a28734, State: Initialized, Role: FOLLOWER
I20260811 18:48:14.233526 20389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34543 } }
I20260811 18:48:14.233606 20389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:14.233646 20389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:14.233690 20389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:14.234292 20389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34543 } }
I20260811 18:48:14.234400 20389 leader_election.cc:302] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 875a2ab3a78b47ff8ff36fbdb3a28734; no voters:
I20260811 18:48:14.234623 20403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:14.234597 20389 leader_election.cc:288] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:14.234786 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.229424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.9.215.62:34543)
0811 18:48:14.229618 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.229627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.229660 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:48:14.230245 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.230256 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.230372 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.230535 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:14.230811 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.231805 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.232598 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.233020 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.233148 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.233927 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:14.233932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:14.233932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:14.234123 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:14.234163 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:14.234175 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:14.234230 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:14.234308 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 18:48:14.234500 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260811 18:48:14.234954 20389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:14.234789 20403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [term 1 LEADER]: Becoming Leader. State: Replica: 875a2ab3a78b47ff8ff36fbdb3a28734, State: Running, Role: LEADER
I20260811 18:48:14.235185 20403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34543 } }
I20260811 18:48:14.235620 20405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34543 } } }
I20260811 18:48:14.235702 20405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.235885 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:48:14.236088 20404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 875a2ab3a78b47ff8ff36fbdb3a28734. Latest consensus state: current_term: 1 leader_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "875a2ab3a78b47ff8ff36fbdb3a28734" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34543 } } }
I20260811 18:48:14.236143 20404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734 [sys.catalog]: This master's current role is: LEADER
W20260811 18:48:14.236896 20430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:48:14.236954 20430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:48:14.237006 20421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:14.237169 20421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:14.238909 20421 catalog_manager.cc:1347] Generated new cluster ID: 5994b1fd340c4671b4f2b19937cb84df
I20260811 18:48:14.238983 20421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:14.248981 20421 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:14.249827 20421 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:14.269930 20421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 875a2ab3a78b47ff8ff36fbdb3a28734: Generated new TSK 0
I20260811 18:48:14.270218 20421 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:48:14.292740 20448 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 18:48:14.292922 20448 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:48:14.293219 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.276713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.9.215.62:34543)
0811 18:48:14.276772 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.276774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.276927 (+ 153us) server_negotiation.cc:407] Connection header received
0811 18:48:14.277004 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.277005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.277017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.277079 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:48:14.277275 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.277841 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.279005 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.279251 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.279309 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.279616 (+ 307us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:48:14.292789 (+ 13173us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:48:14.293148 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43930: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 18:48:14.293272 20394 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43930: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 18:48:14.293787 20446 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:14.276578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34543 (local address 127.0.0.1:43930)
0811 18:48:14.276800 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:14.276814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:14.276912 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:14.277103 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:14.277104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:14.277106 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:48:14.277183 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:14.277185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.277928 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.277929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:14.278909 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:14.278914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.279346 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.279348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:14.279418 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.279479 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:48:14.293154 (+ 13675us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:48:14.293321 (+ 167us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.215.62:34543: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.215.62:34543: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40153/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 18:48:14.294237 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20255
2026-08-11T18:48:14Z chronyd exiting
I20260811 18:48:14.350749 10076 test_util.cc:175] -----------------------------------------------
I20260811 18:48:14.350842 10076 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474068101683-10076-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (259 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 18:48:14.351768 10076 mini_oidc.cc:152] Starting JWKS server
I20260811 18:48:14.352484 10076 webserver.cc:460] Webserver started at https://127.0.0.1:45895/ using document root <none> and password file <none>
I20260811 18:48:14.352540 10076 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:48:14.352690 10076 webserver.cc:460] Webserver started at http://127.0.0.1:45711/ using document root <none> and password file <none>
2026-08-11T18:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:14Z Disabled control of system clock
I20260811 18:48:14.362453 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45237
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:45237
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45895/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/testchainca.cert with env {}
W20260811 18:48:14.449085 20463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:14.449357 20463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:14.449393 20463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:14.450661 20463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:14.450716 20463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:14.450732 20463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:14.450749 20463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:14.450764 20463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:48:14.450779 20463 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:48:14.450794 20463 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45895/jwks.json
I20260811 18:48:14.452549 20463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:45237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45895/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:14.452836 20463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:14.453052 20463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:14.458199 20472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.458250 20463 server_base.cc:1034] running on GCE node
W20260811 18:48:14.458191 20475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:14.458185 20471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.458621 20463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:14.458921 20463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:14.460069 20463 hybrid_clock.cc:648] HybridClock initialized: now 1786474094460053 us; error 27 us; skew 500 ppm
I20260811 18:48:14.461380 20463 webserver.cc:460] Webserver started at http://127.9.215.62:35137/ using document root <none> and password file <none>
I20260811 18:48:14.461615 20463 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:14.461666 20463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:14.461796 20463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:14.462877 20463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "c65253f6419f4cbe9386230d9db033a3"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "41d98d7b18d86a6b55ffe1eda265205f"
server_key_iv: "0f7a4aa00c92e6b7d40ff30c53c3fd80"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.463223 20463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "c65253f6419f4cbe9386230d9db033a3"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "41d98d7b18d86a6b55ffe1eda265205f"
server_key_iv: "0f7a4aa00c92e6b7d40ff30c53c3fd80"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.464673 20463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:14.465520 20487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:14.465731 20463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:14.465828 20463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c65253f6419f4cbe9386230d9db033a3"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "41d98d7b18d86a6b55ffe1eda265205f"
server_key_iv: "0f7a4aa00c92e6b7d40ff30c53c3fd80"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.465898 20463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:14.481655 20463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:14.482021 20463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:14.482170 20463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:14.486490 20463 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:45237
I20260811 18:48:14.486536 20597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:45237 every 8 connection(s)
I20260811 18:48:14.486896 20463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:14.487661 20598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:14.488201 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20463
I20260811 18:48:14.488297 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:14.488446 10076 external_mini_cluster.cc:1428] Setting key 6bf3a75132f240417fd5cbc7884f0a75
I20260811 18:48:14.492110 20598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3: Bootstrap starting.
I20260811 18:48:14.492839 20598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:14.493208 20598 log.cc:824] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:14.494066 20598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3: No bootstrap required, opened a new log
I20260811 18:48:14.495203 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.488995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.9.215.62:45237)
0811 18:48:14.489232 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.489242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.489280 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:48:14.489855 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.489867 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.489963 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.490154 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:14.490778 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.492176 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.493030 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.493621 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.493744 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.494438 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:14.494443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:14.494444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:14.494674 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:14.494722 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:14.494740 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:14.494814 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:14.494870 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 18:48:14.494963 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260811 18:48:14.496428 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:48:14.496579 20598 raft_consensus.cc:348] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65253f6419f4cbe9386230d9db033a3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45237 } }
I20260811 18:48:14.496748 20598 raft_consensus.cc:374] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:14.496786 20598 raft_consensus.cc:729] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c65253f6419f4cbe9386230d9db033a3, State: Initialized, Role: FOLLOWER
I20260811 18:48:14.496923 20598 consensus_queue.cc:260] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65253f6419f4cbe9386230d9db033a3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45237 } }
I20260811 18:48:14.496995 20598 raft_consensus.cc:388] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:14.497040 20598 raft_consensus.cc:482] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:14.497094 20598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:14.497849 20598 raft_consensus.cc:504] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65253f6419f4cbe9386230d9db033a3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45237 } }
I20260811 18:48:14.497963 20598 leader_election.cc:302] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c65253f6419f4cbe9386230d9db033a3; no voters:
I20260811 18:48:14.498167 20598 leader_election.cc:288] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:14.498227 20609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:14.498409 20609 raft_consensus.cc:686] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [term 1 LEADER]: Becoming Leader. State: Replica: c65253f6419f4cbe9386230d9db033a3, State: Running, Role: LEADER
I20260811 18:48:14.498450 20598 sys_catalog.cc:564] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:14.498510 20609 consensus_queue.cc:237] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65253f6419f4cbe9386230d9db033a3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45237 } }
I20260811 18:48:14.498904 20610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c65253f6419f4cbe9386230d9db033a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65253f6419f4cbe9386230d9db033a3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45237 } } }
I20260811 18:48:14.498911 20611 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c65253f6419f4cbe9386230d9db033a3. Latest consensus state: current_term: 1 leader_uuid: "c65253f6419f4cbe9386230d9db033a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65253f6419f4cbe9386230d9db033a3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45237 } } }
I20260811 18:48:14.499017 20610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.499019 20611 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.499267 20621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:14.499850 20621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:14.501519 20621 catalog_manager.cc:1347] Generated new cluster ID: 1865898dfdc94b409d81ee7ee9754e1c
I20260811 18:48:14.501577 20621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:14.509095 20621 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:14.509852 20621 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:14.518478 20621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c65253f6419f4cbe9386230d9db033a3: Generated new TSK 0
I20260811 18:48:14.518719 20621 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:48:14.543743 20661 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 18:48:14.543843 20661 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:48:14.544209 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.528422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.9.215.62:45237)
0811 18:48:14.528477 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.528479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.528661 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:48:14.528743 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.528745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.528758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.528832 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:48:14.529025 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.529583 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.530759 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.530977 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.531036 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.531255 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:48:14.543976 (+ 12721us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:48:14.544144 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44202: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 18:48:14.544260 20602 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44202: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 18:48:14.544413 20657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:14.528301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45237 (local address 127.0.0.1:44202)
0811 18:48:14.528547 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:14.528559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:14.528652 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:14.528850 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:14.528851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:14.528852 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:48:14.528925 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:14.528928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.529672 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.529673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:14.530642 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:14.530647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.531058 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.531059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:14.531124 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.531132 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:48:14.544127 (+ 12995us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:48:14.544290 (+ 163us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.215.62:45237: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":103,"threads_started":1}
I20260811 18:48:14.545042 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20463
2026-08-11T18:48:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 18:48:14.593458 10076 mini_oidc.cc:152] Starting JWKS server
I20260811 18:48:14.594138 10076 webserver.cc:460] Webserver started at https://127.0.0.1:36671/ using document root <none> and password file <none>
I20260811 18:48:14.594183 10076 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:48:14.594303 10076 webserver.cc:460] Webserver started at http://127.0.0.1:35995/ using document root <none> and password file <none>
2026-08-11T18:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:14Z Disabled control of system clock
I20260811 18:48:14.602869 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43383
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:43383
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:48:14.685717 20669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:14.685949 20669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:14.685973 20669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:14.687150 20669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:14.687187 20669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:14.687198 20669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:14.687211 20669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:14.687220 20669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:48:14.687232 20669 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:48:14.687247 20669 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36671/jwks.json
I20260811 18:48:14.688651 20669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:43383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:14.688892 20669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:14.689168 20669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:14.694334 20683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:14.694339 20677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:14.694339 20676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.694363 20669 server_base.cc:1034] running on GCE node
I20260811 18:48:14.694927 20669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:14.695246 20669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:14.696431 20669 hybrid_clock.cc:648] HybridClock initialized: now 1786474094696411 us; error 31 us; skew 500 ppm
I20260811 18:48:14.697929 20669 webserver.cc:460] Webserver started at http://127.9.215.62:37105/ using document root <none> and password file <none>
I20260811 18:48:14.698189 20669 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:14.698239 20669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:14.698342 20669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:14.699533 20669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "830cfc08894f4c4eb508b033f28e7c1e"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "23443d1874b51cf245b8a86cc3ace236"
server_key_iv: "99c2507e053272c88de9f5d557653426"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.699899 20669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "830cfc08894f4c4eb508b033f28e7c1e"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "23443d1874b51cf245b8a86cc3ace236"
server_key_iv: "99c2507e053272c88de9f5d557653426"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.701320 20669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:14.702175 20694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:14.702405 20669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:14.702504 20669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "830cfc08894f4c4eb508b033f28e7c1e"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "23443d1874b51cf245b8a86cc3ace236"
server_key_iv: "99c2507e053272c88de9f5d557653426"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.702585 20669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:14.723639 20669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:14.723960 20669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:14.724090 20669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:14.727993 20669 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:43383
I20260811 18:48:14.728078 20796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:43383 every 8 connection(s)
I20260811 18:48:14.728381 20669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:14.729058 20797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:14.731947 20797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e: Bootstrap starting.
I20260811 18:48:14.732542 20797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:14.732805 20797 log.cc:824] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:14.733510 20797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e: No bootstrap required, opened a new log
I20260811 18:48:14.735119 20797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830cfc08894f4c4eb508b033f28e7c1e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43383 } }
I20260811 18:48:14.735271 20797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:14.735304 20797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830cfc08894f4c4eb508b033f28e7c1e, State: Initialized, Role: FOLLOWER
I20260811 18:48:14.735477 20797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830cfc08894f4c4eb508b033f28e7c1e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43383 } }
I20260811 18:48:14.735564 20797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:14.735631 20797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:14.735687 20797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:14.736279 20797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830cfc08894f4c4eb508b033f28e7c1e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43383 } }
I20260811 18:48:14.736383 20797 leader_election.cc:302] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830cfc08894f4c4eb508b033f28e7c1e; no voters:
I20260811 18:48:14.736584 20797 leader_election.cc:288] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:14.736694 20806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:14.736860 20806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [term 1 LEADER]: Becoming Leader. State: Replica: 830cfc08894f4c4eb508b033f28e7c1e, State: Running, Role: LEADER
I20260811 18:48:14.736886 20797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:14.736967 20806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830cfc08894f4c4eb508b033f28e7c1e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43383 } }
I20260811 18:48:14.737353 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830cfc08894f4c4eb508b033f28e7c1e. Latest consensus state: current_term: 1 leader_uuid: "830cfc08894f4c4eb508b033f28e7c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830cfc08894f4c4eb508b033f28e7c1e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43383 } } }
I20260811 18:48:14.737442 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.737624 20807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "830cfc08894f4c4eb508b033f28e7c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830cfc08894f4c4eb508b033f28e7c1e" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43383 } } }
I20260811 18:48:14.737682 20807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.737818 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20669
I20260811 18:48:14.737900 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:14.738066 10076 external_mini_cluster.cc:1428] Setting key 096e17325e9f36d86f928246e986c81c
I20260811 18:48:14.738323 20822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:14.738961 20822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:14.740675 20822 catalog_manager.cc:1347] Generated new cluster ID: 9aa8cdd076c14e178d72e8aeed6c6d16
I20260811 18:48:14.740734 20822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:14.743988 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.738628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.9.215.62:43383)
0811 18:48:14.738816 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.738824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.738850 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:48:14.738951 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.738957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.739019 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.739183 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:14.739412 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.740620 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.741738 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.742362 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.742476 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.743240 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:14.743245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:14.743245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:14.743480 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:14.743522 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:14.743537 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:14.743597 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:14.743690 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 18:48:14.743770 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260811 18:48:14.744849 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:48:14.754158 20822 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:14.754742 20822 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:14.759603 20822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 830cfc08894f4c4eb508b033f28e7c1e: Generated new TSK 0
I20260811 18:48:14.759801 20822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:14.773860 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.770537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.9.215.62:43383)
0811 18:48:14.770588 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.770590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.770726 (+ 136us) server_negotiation.cc:407] Connection header received
0811 18:48:14.770814 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.770815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.770828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.770891 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:48:14.771025 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.771606 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.772369 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.772581 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.772637 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.772828 (+ 191us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:48:14.773648 (+ 820us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:48:14.773783 (+ 135us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 18:48:14.773938 20857 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:14.770475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43383 (local address 127.0.0.1:42638)
0811 18:48:14.770661 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:14.770671 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:14.770735 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:14.770893 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:14.770894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:14.770896 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:48:14.770965 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:14.770968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.771686 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.771687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:14.772286 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:14.772288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.772658 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.772659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:14.772726 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.772732 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:48:14.773746 (+ 1014us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:48:14.773845 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.215.62:43383: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
W20260811 18:48:14.773940 20841 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260811 18:48:14.774406 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20669
E20260811 18:48:14.799819 20863 webserver.cc:533] Webserver: sslize failed:
E20260811 18:48:14.799916 20863 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T18:48:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T18:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:14Z Disabled control of system clock
I20260811 18:48:14.838315 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:39767
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:39767
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/testchainca.cert with env {}
W20260811 18:48:14.922158 20871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:14.922407 20871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:14.922441 20871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:14.923800 20871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:14.923852 20871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:14.923869 20871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:14.923885 20871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:14.923899 20871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:48:14.923916 20871 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:48:14.923935 20871 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 18:48:14.925609 20871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:39767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:39767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:14.925896 20871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:14.926133 20871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:14.931780 20890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:14.931764 20882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.931818 20871 server_base.cc:1034] running on GCE node
W20260811 18:48:14.931761 20884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:14.932224 20871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:14.932547 20871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:14.933699 20871 hybrid_clock.cc:648] HybridClock initialized: now 1786474094933684 us; error 33 us; skew 500 ppm
I20260811 18:48:14.935051 20871 webserver.cc:460] Webserver started at http://127.9.215.62:33505/ using document root <none> and password file <none>
I20260811 18:48:14.935302 20871 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:14.935350 20871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:14.935494 20871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:14.936669 20871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "0a5ccb17bd6c4c3ea109a2208035e90f"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "29686e28d1c31727e5f51e0275a6909f"
server_key_iv: "6a015e72d7736ce6cbb7c3b552c947a4"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.937013 20871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "0a5ccb17bd6c4c3ea109a2208035e90f"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "29686e28d1c31727e5f51e0275a6909f"
server_key_iv: "6a015e72d7736ce6cbb7c3b552c947a4"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.938374 20871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:14.939201 20906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:14.939399 20871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:14.939518 20871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "0a5ccb17bd6c4c3ea109a2208035e90f"
format_stamp: "Formatted at 2026-08-11 18:48:14 on dist-test-slave-pww0"
server_key: "29686e28d1c31727e5f51e0275a6909f"
server_key_iv: "6a015e72d7736ce6cbb7c3b552c947a4"
server_key_version: "encryptionkey@0"
I20260811 18:48:14.939589 20871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:14.966673 20871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:14.967056 20871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:14.967211 20871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:14.971532 20871 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:39767
I20260811 18:48:14.971596 21013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:39767 every 8 connection(s)
I20260811 18:48:14.971949 20871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:14.972991 21015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:14.974042 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20871
I20260811 18:48:14.974170 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:14.974354 10076 external_mini_cluster.cc:1428] Setting key 03424402fbe93d0dcfdf34285f8cbab5
I20260811 18:48:14.976306 21015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f: Bootstrap starting.
I20260811 18:48:14.977003 21015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:14.977319 21015 log.cc:824] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:14.978082 21015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f: No bootstrap required, opened a new log
I20260811 18:48:14.980581 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:14.975150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.9.215.62:39767)
0811 18:48:14.975372 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:14.975380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:14.975408 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:48:14.976060 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:14.976069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:14.976142 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:14.976272 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:14.976528 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.977518 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.978372 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:14.978816 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:14.978944 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:14.979701 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:14.979708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:14.979709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:14.979954 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:14.980014 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:14.980033 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:14.980119 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:14.980248 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 18:48:14.980348 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260811 18:48:14.980499 21015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39767 } }
I20260811 18:48:14.980690 21015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:14.980724 21015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5ccb17bd6c4c3ea109a2208035e90f, State: Initialized, Role: FOLLOWER
I20260811 18:48:14.980852 21015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39767 } }
I20260811 18:48:14.980937 21015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:14.980973 21015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:14.981019 21015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:14.981774 21015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39767 } }
I20260811 18:48:14.981871 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:48:14.981899 21015 leader_election.cc:302] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a5ccb17bd6c4c3ea109a2208035e90f; no voters:
I20260811 18:48:14.982091 21015 leader_election.cc:288] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:14.982147 21026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:14.982321 21026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [term 1 LEADER]: Becoming Leader. State: Replica: 0a5ccb17bd6c4c3ea109a2208035e90f, State: Running, Role: LEADER
I20260811 18:48:14.982378 21015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:14.982420 21026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39767 } }
I20260811 18:48:14.982816 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5ccb17bd6c4c3ea109a2208035e90f. Latest consensus state: current_term: 1 leader_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39767 } } }
I20260811 18:48:14.982895 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.983116 21037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:14.983243 21029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5ccb17bd6c4c3ea109a2208035e90f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39767 } } }
I20260811 18:48:14.983335 21029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:14.983680 21037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:14.985503 21037 catalog_manager.cc:1347] Generated new cluster ID: 8165f7da07ba4e15aa9c22840017242e
I20260811 18:48:14.985559 21037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:15.015895 21037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:15.016683 21037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:15.022864 21037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a5ccb17bd6c4c3ea109a2208035e90f: Generated new TSK 0
I20260811 18:48:15.023146 21037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:15.596315 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:15.035897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.9.215.62:39767)
0811 18:48:15.035946 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:15.035948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:15.036059 (+ 111us) server_negotiation.cc:407] Connection header received
0811 18:48:15.036107 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:15.036108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:15.036123 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:15.036204 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:48:15.036340 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:15.036861 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:15.037996 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:15.038209 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:15.038265 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:15.038481 (+ 216us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:48:15.596046 (+557565us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:48:15.596243 (+ 197us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47786: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 18:48:15.596398 21023 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47786: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 18:48:15.596462 21068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:15.035807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:39767 (local address 127.0.0.1:47786)
0811 18:48:15.035956 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:15.035967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:15.036043 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:15.036212 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:15.036213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:15.036214 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:48:15.036282 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:15.036285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:15.036945 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:15.036946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:15.037890 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:15.037894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:15.038285 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:15.038286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:15.038350 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:15.038356 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:48:15.596226 (+557870us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:48:15.596371 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.215.62:39767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260811 18:48:15.596804 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20871
2026-08-11T18:48:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (793 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:15 dist-test-slave-pww0 krb5kdc[21094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:15 dist-test-slave-pww0 krb5kdc[21094](info): set up 2 sockets
Aug 11 18:48:15 dist-test-slave-pww0 krb5kdc[21094](info): commencing operation
krb5kdc: starting...
W20260811 18:48:17.014757 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:17.145905 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34051
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:34051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:17.232306 21120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:17.232528 21120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:17.232551 21120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:17.233816 21120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:17.233865 21120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:17.233878 21120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:17.233889 21120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:17.233901 21120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:17.235528 21120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:34051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:17.235803 21120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:17.236028 21120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:17.241329 21136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.241405 21120 server_base.cc:1034] running on GCE node
W20260811 18:48:17.241525 21137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:17.241338 21140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.241958 21120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:17.242282 21120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:17.243443 21120 hybrid_clock.cc:648] HybridClock initialized: now 1786474097243425 us; error 30 us; skew 500 ppm
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.246191 21120 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:17.246767 21120 webserver.cc:460] Webserver started at http://127.9.215.62:46635/ using document root <none> and password file <none>
I20260811 18:48:17.247013 21120 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:17.247068 21120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:17.247211 21120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:17.248409 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "28b7d91e81a441e4a60f001595657ff9"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "a803ce98a17916c7202e71371afeeb35"
server_key_iv: "ef55364246494680d99b907c089893cd"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.248782 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "28b7d91e81a441e4a60f001595657ff9"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "a803ce98a17916c7202e71371afeeb35"
server_key_iv: "ef55364246494680d99b907c089893cd"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.250255 21120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:17.251071 21157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.251262 21120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:17.251346 21120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "28b7d91e81a441e4a60f001595657ff9"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "a803ce98a17916c7202e71371afeeb35"
server_key_iv: "ef55364246494680d99b907c089893cd"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.251420 21120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:17.267532 21120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:17.269850 21120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:17.270005 21120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:17.274382 21120 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:34051
I20260811 18:48:17.274428 21248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:34051 every 8 connection(s)
I20260811 18:48:17.274801 21120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:17.275765 21252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:17.278215 21252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9: Bootstrap starting.
I20260811 18:48:17.278858 21252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:17.279150 21252 log.cc:824] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:17.279950 21252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9: No bootstrap required, opened a new log
I20260811 18:48:17.281641 21252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b7d91e81a441e4a60f001595657ff9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34051 } }
I20260811 18:48:17.281807 21252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:17.281829 21252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b7d91e81a441e4a60f001595657ff9, State: Initialized, Role: FOLLOWER
I20260811 18:48:17.281981 21252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b7d91e81a441e4a60f001595657ff9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34051 } }
I20260811 18:48:17.282047 21252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:17.282071 21252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:17.282119 21252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:17.282752 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21120
I20260811 18:48:17.282778 21252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b7d91e81a441e4a60f001595657ff9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34051 } }
I20260811 18:48:17.282857 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:17.282884 21252 leader_election.cc:302] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28b7d91e81a441e4a60f001595657ff9; no voters:
I20260811 18:48:17.283062 10076 external_mini_cluster.cc:1428] Setting key 8229e4b28b533ced0a045b1d30d4c11f
I20260811 18:48:17.283078 21252 leader_election.cc:288] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:17.283136 21258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:17.283357 21252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:17.283684 21258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [term 1 LEADER]: Becoming Leader. State: Replica: 28b7d91e81a441e4a60f001595657ff9, State: Running, Role: LEADER
I20260811 18:48:17.283854 21258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b7d91e81a441e4a60f001595657ff9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34051 } }
I20260811 18:48:17.284237 21258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b7d91e81a441e4a60f001595657ff9. Latest consensus state: current_term: 1 leader_uuid: "28b7d91e81a441e4a60f001595657ff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b7d91e81a441e4a60f001595657ff9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34051 } } }
I20260811 18:48:17.284387 21258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:17.284428 21262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28b7d91e81a441e4a60f001595657ff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b7d91e81a441e4a60f001595657ff9" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34051 } } }
I20260811 18:48:17.284484 21262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:17.284832 21282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:17.285759 21282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:17.287935 21282 catalog_manager.cc:1347] Generated new cluster ID: 77f08b5a92a24b61a5ae6189acf252b6
I20260811 18:48:17.288000 21282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:17.293823 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.283781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.9.215.62:34051)
0811 18:48:17.283984 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.283995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.284035 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:48:17.284189 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.284199 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.284264 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.284419 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.285497 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.286484 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.287550 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.288155 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.288253 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.290572 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.290578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.290578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.290593 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.292279 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.292627 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.292631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.292632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.292743 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.292923 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.292924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.292924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.293200 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.293375 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.293448 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.293602 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260811 18:48:17.297433 21282 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:17.298008 21282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:17.303041 21282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28b7d91e81a441e4a60f001595657ff9: Generated new TSK 0
I20260811 18:48:17.303262 21282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:17.335043 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34051
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:17.422214 21299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:17.422459 21299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:17.422497 21299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:17.423779 21299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:17.423832 21299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:17.423871 21299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:17.425505 21299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:17.425773 21299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:17.426007 21299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:17.431295 21306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:17.431258 21313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:17.431267 21305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.431406 21299 server_base.cc:1034] running on GCE node
I20260811 18:48:17.431830 21299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:17.432085 21299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:17.433256 21299 hybrid_clock.cc:648] HybridClock initialized: now 1786474097433242 us; error 32 us; skew 500 ppm
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.436008 21299 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:17.436488 21299 webserver.cc:460] Webserver started at http://127.9.215.1:32821/ using document root <none> and password file <none>
I20260811 18:48:17.436710 21299 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:17.436761 21299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:17.436893 21299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:17.438022 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "1bfc1e53bcf94788b0c90d612c545f31"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "400e5e114d1560a222fd3cfca52ebb60"
server_key_iv: "52d35a5332c3b484cb99f8f3309fff08"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.438367 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "1bfc1e53bcf94788b0c90d612c545f31"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "400e5e114d1560a222fd3cfca52ebb60"
server_key_iv: "52d35a5332c3b484cb99f8f3309fff08"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.439863 21299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:48:17.440773 21326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.440994 21299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:17.441111 21299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "1bfc1e53bcf94788b0c90d612c545f31"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "400e5e114d1560a222fd3cfca52ebb60"
server_key_iv: "52d35a5332c3b484cb99f8f3309fff08"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.441185 21299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:17.451054 21299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:17.453651 21299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:17.453823 21299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:17.454077 21299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:17.454497 21299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:17.454591 21299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.454627 21299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:17.454641 21299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.460480 21299 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:44821
I20260811 18:48:17.460573 21566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:44821 every 8 connection(s)
I20260811 18:48:17.460891 21299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:17.461448 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21299
I20260811 18:48:17.461545 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:17.461750 10076 external_mini_cluster.cc:1428] Setting key 6a24743b673f4a8808d716d68f04914a
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:48:17.472582 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.461753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:35121 (local address 127.9.215.62:34051)
0811 18:48:17.461828 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.461830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.462047 (+ 217us) server_negotiation.cc:407] Connection header received
0811 18:48:17.463112 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.463113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.463129 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.463200 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.463936 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.464552 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.466030 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.466253 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.466318 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.469072 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.469073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.469074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.469083 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.470715 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.471313 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.471314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.471314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.471384 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.471713 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.471715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.471715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.471851 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.471919 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.472327 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.472448 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:48:17.472720 21573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.461650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34051 (local address 127.9.215.1:35121)
0811 18:48:17.461903 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:17.461952 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:17.462975 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:17.463276 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:17.463280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:17.463619 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:17.463835 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:17.463840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.464669 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.464670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:17.465914 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:17.465919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.466346 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.466347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:17.466495 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.467049 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:17.467062 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:17.468919 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:17.470825 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:17.470840 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:17.470845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:17.471197 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:17.471495 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:17.471496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:17.471496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:17.471627 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:17.471949 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:17.471958 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:17.472153 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 18:48:17.472260 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:48:17.472471 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":110,"threads_started":1}
I20260811 18:48:17.473356 21567 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34051
I20260811 18:48:17.473480 21567 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:17.473657 21567 heartbeater.cc:511] Master 127.9.215.62:34051 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
I20260811 18:48:17.474196 21188 ts_manager.cc:194] Registered new tserver with Master: 1bfc1e53bcf94788b0c90d612c545f31 (127.9.215.1:44821)
I20260811 18:48:17.474830 21188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:35121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:17.490716 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34051
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:17.576840 21579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:17.577088 21579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:17.577128 21579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:17.578435 21579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:17.578490 21579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:17.578528 21579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:17.580274 21579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:17.580559 21579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:17.580796 21579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:17.586244 21587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.586369 21579 server_base.cc:1034] running on GCE node
W20260811 18:48:17.586335 21593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:17.586349 21586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.586880 21579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:17.587139 21579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:17.588294 21579 hybrid_clock.cc:648] HybridClock initialized: now 1786474097588272 us; error 36 us; skew 500 ppm
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.591123 21579 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:17.591715 21579 webserver.cc:460] Webserver started at http://127.9.215.2:34727/ using document root <none> and password file <none>
I20260811 18:48:17.591950 21579 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:17.592003 21579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:17.592151 21579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:17.593257 21579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "2b33d554698148888a487baaacc6d14e"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "e1b2dccee5b5e88d304d60bcd7f4ab66"
server_key_iv: "037a5659c7bac8d353c88d4101e94c25"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.593613 21579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "2b33d554698148888a487baaacc6d14e"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "e1b2dccee5b5e88d304d60bcd7f4ab66"
server_key_iv: "037a5659c7bac8d353c88d4101e94c25"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.595047 21579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:17.595920 21606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.596117 21579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:48:17.596220 21579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "2b33d554698148888a487baaacc6d14e"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "e1b2dccee5b5e88d304d60bcd7f4ab66"
server_key_iv: "037a5659c7bac8d353c88d4101e94c25"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.596293 21579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:17.641757 21579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:17.644292 21579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:17.644469 21579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:17.644711 21579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:17.645037 21579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:17.645073 21579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.645103 21579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:17.645129 21579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.651057 21579 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:35817
I20260811 18:48:17.651106 21830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:35817 every 8 connection(s)
I20260811 18:48:17.651496 21579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:17.657910 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21579
I20260811 18:48:17.658034 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:17.658228 10076 external_mini_cluster.cc:1428] Setting key cb98f6e4cf9fc2a71a674a96fdde814c
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:17.661953 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.652273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:57189 (local address 127.9.215.62:34051)
0811 18:48:17.652321 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.652323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.652582 (+ 259us) server_negotiation.cc:407] Connection header received
0811 18:48:17.653519 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.653521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.653534 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.653616 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.654433 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.654966 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.656166 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.656390 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.656466 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.658885 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.658887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.658888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.658897 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.660371 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.660900 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.660901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.660902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.660997 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.661253 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.661253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.661254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.661402 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.661466 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.661766 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.661863 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:17.662062 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.652240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34051 (local address 127.9.215.2:57189)
0811 18:48:17.652415 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:17.652462 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:17.653386 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:17.653760 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:17.653764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:17.654059 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:17.654309 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:17.654315 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.655076 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.655077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:17.656061 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:17.656064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.656504 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.656505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:17.656660 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.657191 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:17.657200 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:17.658751 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:17.660460 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:17.660472 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:17.660475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:17.660815 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:17.661066 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:17.661067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:17.661067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:17.661195 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:17.661463 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:17.661470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:17.661640 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 18:48:17.661729 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:48:17.661870 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260811 18:48:17.662571 21831 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34051
I20260811 18:48:17.662681 21831 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:17.662868 21831 heartbeater.cc:511] Master 127.9.215.62:34051 requested a full tablet report, sending...
I20260811 18:48:17.663272 21188 ts_manager.cc:194] Registered new tserver with Master: 2b33d554698148888a487baaacc6d14e (127.9.215.2:35817)
I20260811 18:48:17.663843 21188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:57189
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:17.687129 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34051
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:17.772145 21841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:17.772382 21841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:17.772418 21841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:17.773662 21841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:17.773715 21841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:17.773754 21841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:17.775502 21841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:17.775782 21841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:17.776014 21841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:17.781245 21856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.781472 21841 server_base.cc:1034] running on GCE node
W20260811 18:48:17.781234 21860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:17.781399 21858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:17.781823 21841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:17.782105 21841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:17.783224 21841 hybrid_clock.cc:648] HybridClock initialized: now 1786474097783196 us; error 33 us; skew 500 ppm
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.786118 21841 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:17.786597 21841 webserver.cc:460] Webserver started at http://127.9.215.3:46283/ using document root <none> and password file <none>
I20260811 18:48:17.786844 21841 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:17.786897 21841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:17.787027 21841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:17.788213 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "fb25c8478a89438ba0db35f1763d9b8d"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "38b62615286935b56b84f8411e3b0b19"
server_key_iv: "9e0bf25532fe77ec1685871c0e6a8b55"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.788560 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "fb25c8478a89438ba0db35f1763d9b8d"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "38b62615286935b56b84f8411e3b0b19"
server_key_iv: "9e0bf25532fe77ec1685871c0e6a8b55"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.790007 21841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:48:17.790944 21870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.791307 21841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:17.791419 21841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "fb25c8478a89438ba0db35f1763d9b8d"
format_stamp: "Formatted at 2026-08-11 18:48:17 on dist-test-slave-pww0"
server_key: "38b62615286935b56b84f8411e3b0b19"
server_key_iv: "9e0bf25532fe77ec1685871c0e6a8b55"
server_key_version: "encryptionkey@0"
I20260811 18:48:17.791620 21841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:17.808722 21841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:17.810974 21841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:17.811158 21841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:17.811430 21841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:17.811861 21841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:17.811903 21841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.811939 21841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:17.811967 21841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:17.818415 21841 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:39351
I20260811 18:48:17.818483 22075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:39351 every 8 connection(s)
I20260811 18:48:17.818828 21841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:17.823567 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21841
I20260811 18:48:17.823652 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:17.823832 10076 external_mini_cluster.cc:1428] Setting key 129c0c3f02431f9f41aed26b34112133
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:17.829104 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.819648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:51593 (local address 127.9.215.62:34051)
0811 18:48:17.819709 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.819711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.819963 (+ 252us) server_negotiation.cc:407] Connection header received
0811 18:48:17.820995 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.820997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.821012 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.821082 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.821846 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.822361 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.823526 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.823785 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.823844 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.826384 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.826386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.826386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.826394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.827787 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.828263 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.828264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.828265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.828326 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.828547 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.828548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.828548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.828655 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.828702 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.828964 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.829034 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:48:17.829298 22083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.819632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34051 (local address 127.9.215.3:51593)
0811 18:48:17.819828 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:17.819870 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:17.820866 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:17.821189 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:17.821193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:17.821496 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:17.821741 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:17.821746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.822466 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.822468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:17.823355 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:17.823359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.823881 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.823883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:17.824053 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.824620 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:17.824632 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:17.826283 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:17.827888 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:17.827897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:17.827900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:17.828179 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:17.828378 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:17.828378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:17.828379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:17.828497 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:17.828698 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:17.828702 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:17.828857 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 18:48:17.828940 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:48:17.829076 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260811 18:48:17.829814 22079 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34051
I20260811 18:48:17.829919 22079 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:17.830116 22079 heartbeater.cc:511] Master 127.9.215.62:34051 requested a full tablet report, sending...
I20260811 18:48:17.830516 21188 ts_manager.cc:194] Registered new tserver with Master: fb25c8478a89438ba0db35f1763d9b8d (127.9.215.3:39351)
I20260811 18:48:17.830911 21188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:51593
I20260811 18:48:17.836145 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/test-admin.keytab.
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.852201 10076 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 18:48:17.854346 22090 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:17.853372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.9.215.1:44821)
0811 18:48:17.853556 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.853562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.853573 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:17.853602 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.853604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.853619 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.853694 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.854248 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54096: BlockingRecv error: recv got EOF from 127.0.0.1:54096 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
W20260811 18:48:17.854529 10076 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.855661 10076 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.856755 10076 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:48:17.865413 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.857512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.9.215.1:44821)
0811 18:48:17.857579 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.857581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.857590 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:17.857618 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.857619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.857658 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.857708 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.858121 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.858888 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.859693 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.860046 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.860110 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.862519 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.862533 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.862533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.862543 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.864197 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.864555 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.864559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.864559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.864646 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.864823 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.864824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.864824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.864979 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.865122 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.865212 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.865303 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 18:48:17.867432 22090 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:17.866776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.9.215.1:44821)
0811 18:48:17.866827 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.866829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.866837 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:17.866874 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.866875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.866885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.866925 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.867383 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54122: BlockingRecv error: recv got EOF from 127.0.0.1:54122 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 18:48:17.867501 10076 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.868467 10076 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.869285 10076 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:48:17.876557 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.869842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.9.215.1:44821)
0811 18:48:17.869895 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.869897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.869938 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:48:17.870004 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.870006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.870018 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.870062 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.870445 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.870939 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.871650 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.871853 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.871910 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.873935 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.873936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.873936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.873945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.875530 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.875856 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.875857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.875857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.875911 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.876068 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.876069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.876069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.876172 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.876257 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.876367 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.876457 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 18:48:17.878093 10076 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474068101683-10076-0/krb5kdc/krb5cc' of type FILE
W20260811 18:48:17.878087 22090 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:17.877529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.9.215.1:44821)
0811 18:48:17.877589 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.877591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.877619 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:48:17.877672 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.877672 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.877681 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.877731 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.878015 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54132: BlockingRecv error: recv got EOF from 127.0.0.1:54132 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.879148 10076 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:17.880115 10076 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:48:17 dist-test-slave-pww0 krb5kdc[21094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:48:17.887997 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:17.880688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.9.215.1:44821)
0811 18:48:17.880768 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:17.880770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:17.880780 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:17.880822 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:17.880823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:17.880832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:17.880888 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:17.881221 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.881964 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.882932 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:17.883129 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:17.883199 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:17.885560 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:17.885562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:17.885562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:17.885570 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:17.886865 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.887243 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.887244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.887245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.887299 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:17.887516 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:17.887517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:17.887517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:17.887616 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:17.887712 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:17.887817 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 18:48:17.887900 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:48:17.888598 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21299
I20260811 18:48:17.908952 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21579
I20260811 18:48:17.963135 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21841
I20260811 18:48:17.976186 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21120
2026-08-11T18:48:17Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2200 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:18 dist-test-slave-pww0 krb5kdc[22117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:18 dist-test-slave-pww0 krb5kdc[22117](info): set up 2 sockets
Aug 11 18:48:18 dist-test-slave-pww0 krb5kdc[22117](info): commencing operation
krb5kdc: starting...
W20260811 18:48:19.221827 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab.
Entry for principal oryx/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab.
I20260811 18:48:19.356633 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37933
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab
--principal=oryx/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:19.441844 22150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:19.442098 22150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:19.442126 22150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:19.443522 22150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:19.443578 22150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:19.443591 22150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:19.443603 22150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:19.443624 22150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:19.445363 22150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab
--principal=oryx/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:19.445662 22150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:19.445909 22150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:19.451309 22167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:19.451319 22161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:19.451334 22158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:19.451529 22150 server_base.cc:1034] running on GCE node
I20260811 18:48:19.451987 22150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:19.452368 22150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:19.453541 22150 hybrid_clock.cc:648] HybridClock initialized: now 1786474099453496 us; error 60 us; skew 500 ppm
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:19.460150 22150 init.cc:377] Logged in from keytab as oryx/127.9.215.62@KRBTEST.COM (short username oryx)
I20260811 18:48:19.460701 22150 webserver.cc:460] Webserver started at http://127.9.215.62:44383/ using document root <none> and password file <none>
I20260811 18:48:19.460948 22150 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:19.461000 22150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:19.461158 22150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:19.462296 22150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "5abd1ef3b2954060a1020004d81f4bf1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "656c3d318e608aeef4a9e30010e84716"
server_key_iv: "460d7a3a364ad13bafe756f90cbde525"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.462662 22150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "5abd1ef3b2954060a1020004d81f4bf1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "656c3d318e608aeef4a9e30010e84716"
server_key_iv: "460d7a3a364ad13bafe756f90cbde525"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.464244 22150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:19.465253 22179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.465560 22150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:19.465704 22150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "5abd1ef3b2954060a1020004d81f4bf1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "656c3d318e608aeef4a9e30010e84716"
server_key_iv: "460d7a3a364ad13bafe756f90cbde525"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.465785 22150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:19.489547 22150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:19.492003 22150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:19.492167 22150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:19.496379 22150 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:37933
I20260811 18:48:19.496424 22277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:37933 every 8 connection(s)
I20260811 18:48:19.496801 22150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:19.497808 22281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:19.500252 22281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Bootstrap starting.
I20260811 18:48:19.500857 22281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:19.501129 22281 log.cc:824] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:19.501996 22281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: No bootstrap required, opened a new log
I20260811 18:48:19.503046 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22150
I20260811 18:48:19.503150 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:19.503427 10076 external_mini_cluster.cc:1428] Setting key 4f46171ba44aa0c4de83c92a3ac26d3c
I20260811 18:48:19.503961 22281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } }
I20260811 18:48:19.504221 22281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:19.504282 22281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abd1ef3b2954060a1020004d81f4bf1, State: Initialized, Role: FOLLOWER
I20260811 18:48:19.504485 22281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } }
I20260811 18:48:19.504655 22281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:19.504750 22281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:19.504817 22281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:19.505946 22281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } }
I20260811 18:48:19.506047 22281 leader_election.cc:302] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5abd1ef3b2954060a1020004d81f4bf1; no voters:
I20260811 18:48:19.506229 22281 leader_election.cc:288] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:19.506258 22289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:19.506366 22289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 LEADER]: Becoming Leader. State: Replica: 5abd1ef3b2954060a1020004d81f4bf1, State: Running, Role: LEADER
I20260811 18:48:19.506457 22289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } }
I20260811 18:48:19.506872 22289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } }
I20260811 18:48:19.506951 22289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:19.507084 22289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5abd1ef3b2954060a1020004d81f4bf1. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } }
I20260811 18:48:19.506518 22281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:19.507210 22289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:19.507203 22297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:19.508112 22297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:19.510005 22297 catalog_manager.cc:1347] Generated new cluster ID: b6358a463d964b94bea1b908646655ad
I20260811 18:48:19.510066 22297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:19.514379 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:19.504059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38802 (local address 127.9.215.62:37933)
0811 18:48:19.504245 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:19.504254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:19.504283 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:48:19.504413 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:19.504422 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:19.504472 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:19.504599 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:19.505414 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.506507 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.507595 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.508233 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.508348 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:19.511164 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:19.511169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:19.511169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:19.511184 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:19.512887 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:19.513243 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:19.513247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:19.513247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:19.513365 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:19.513546 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:19.513547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:19.513547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:19.513810 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:19.513980 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:19.514051 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 18:48:19.514210 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260811 18:48:19.518944 22297 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:19.519759 22297 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:19.536566 22297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Generated new TSK 0
I20260811 18:48:19.536818 22297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab.
Entry for principal oryx/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab.
I20260811 18:48:19.571034 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37933
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:19.658242 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:19.658497 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:19.658525 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:19.659826 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:19.659876 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:19.659904 22326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:19.661516 22326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:19.661736 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:19.661972 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:19.667346 22338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:19.667559 22326 server_base.cc:1034] running on GCE node
W20260811 18:48:19.667655 22342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:19.667351 22335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:19.667868 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:19.668159 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:19.669339 22326 hybrid_clock.cc:648] HybridClock initialized: now 1786474099669320 us; error 34 us; skew 500 ppm
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:19.672024 22326 init.cc:377] Logged in from keytab as oryx/127.9.215.1@KRBTEST.COM (short username oryx)
I20260811 18:48:19.672560 22326 webserver.cc:460] Webserver started at http://127.9.215.1:41057/ using document root <none> and password file <none>
I20260811 18:48:19.672788 22326 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:19.672842 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:19.672974 22326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:19.674096 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "3948537b697d4d77a7946977d83a5ac1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "0230e43f9b3104c83ccc10d5c56a8bab"
server_key_iv: "f62089f8331ea893cde1f00e6bf0d571"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.674484 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "3948537b697d4d77a7946977d83a5ac1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "0230e43f9b3104c83ccc10d5c56a8bab"
server_key_iv: "f62089f8331ea893cde1f00e6bf0d571"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.676069 22326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:19.677079 22353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.677405 22326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:19.677551 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "3948537b697d4d77a7946977d83a5ac1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "0230e43f9b3104c83ccc10d5c56a8bab"
server_key_iv: "f62089f8331ea893cde1f00e6bf0d571"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.677631 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:19.700366 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:19.702754 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:19.702919 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:19.703183 22326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:19.703558 22326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:19.703600 22326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.703632 22326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:19.703656 22326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.710172 22326 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:43179
I20260811 18:48:19.710233 22566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:43179 every 8 connection(s)
I20260811 18:48:19.710602 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:19.717533 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22326
I20260811 18:48:19.717636 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:19.717849 10076 external_mini_cluster.cc:1428] Setting key 281ace15b11b2ee216e63affef40a181
I20260811 18:48:19.721203 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:19.711448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:49643 (local address 127.9.215.62:37933)
0811 18:48:19.711571 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:19.711573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:19.711765 (+ 192us) server_negotiation.cc:407] Connection header received
0811 18:48:19.712321 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:19.712322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:19.712340 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:19.712423 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:19.713480 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.714017 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.715192 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.715413 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.715550 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:19.718198 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:19.718200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:19.718200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:19.718214 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:19.719766 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:19.720194 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:19.720195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:19.720196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:19.720265 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:19.720567 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:19.720568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:19.720568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:19.720684 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:19.720746 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:19.721064 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 18:48:19.721135 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 18:48:19.721493 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:19.711410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:49643)
0811 18:48:19.711633 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:19.711679 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:19.712194 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:19.712489 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:19.712493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:19.713130 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:19.713352 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:19.713359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.714124 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.714125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:19.715077 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:19.715081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.715562 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.715563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:19.715717 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:19.716408 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:19.716425 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:19.718076 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:19.719862 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:19.719871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:19.719874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:19.720128 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:19.720334 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:19.720336 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:19.720336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:19.720502 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:19.720746 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:19.720753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:19.720931 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 18:48:19.721043 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 18:48:19.721247 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":120,"threads_started":1}
I20260811 18:48:19.722024 22568 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37933
I20260811 18:48:19.722121 22568 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:19.722309 22568 heartbeater.cc:511] Master 127.9.215.62:37933 requested a full tablet report, sending...
I20260811 18:48:19.722865 22209 ts_manager.cc:194] Registered new tserver with Master: 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179)
I20260811 18:48:19.723536 22209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:49643
WARNING: no policy specified for oryx/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab.
Entry for principal oryx/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab.
I20260811 18:48:19.747190 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37933
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:19.831732 22582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:19.831960 22582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:19.831987 22582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:19.833252 22582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:19.833307 22582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:19.833336 22582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:19.835072 22582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:19.835364 22582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:19.835662 22582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:19.841015 22594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:19.841038 22590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:19.841032 22591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:19.841349 22582 server_base.cc:1034] running on GCE node
I20260811 18:48:19.841558 22582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:19.841825 22582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:19.842978 22582 hybrid_clock.cc:648] HybridClock initialized: now 1786474099842959 us; error 37 us; skew 500 ppm
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:19.845589 22582 init.cc:377] Logged in from keytab as oryx/127.9.215.2@KRBTEST.COM (short username oryx)
I20260811 18:48:19.846097 22582 webserver.cc:460] Webserver started at http://127.9.215.2:42011/ using document root <none> and password file <none>
I20260811 18:48:19.846312 22582 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:19.846356 22582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:19.846460 22582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:19.847595 22582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "581b5d0e6de143b4a31bf7e2834108aa"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "42b59fa41f0042abac69656cebf8db96"
server_key_iv: "ced68e950a45508ae2211363824a1a58"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.847982 22582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "581b5d0e6de143b4a31bf7e2834108aa"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "42b59fa41f0042abac69656cebf8db96"
server_key_iv: "ced68e950a45508ae2211363824a1a58"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.849416 22582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:48:19.850206 22611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.850409 22582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:48:19.850504 22582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "581b5d0e6de143b4a31bf7e2834108aa"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "42b59fa41f0042abac69656cebf8db96"
server_key_iv: "ced68e950a45508ae2211363824a1a58"
server_key_version: "encryptionkey@0"
I20260811 18:48:19.850575 22582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:19.858629 22582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:19.861009 22582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:19.861166 22582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:19.861406 22582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:19.861737 22582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:19.861773 22582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.861804 22582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:19.861828 22582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:19.867849 22582 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:33427
I20260811 18:48:19.867906 22826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:33427 every 8 connection(s)
I20260811 18:48:19.868240 22582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:19.873404 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22582
I20260811 18:48:19.873524 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:19.873723 10076 external_mini_cluster.cc:1428] Setting key 689fb58e352a688186434f46c1d2f1bc
Aug 11 18:48:19 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:19.878168 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:19.869010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:59655 (local address 127.9.215.62:37933)
0811 18:48:19.869061 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:19.869063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:19.869263 (+ 200us) server_negotiation.cc:407] Connection header received
0811 18:48:19.869781 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:19.869782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:19.869792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:19.869853 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:19.870837 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.871363 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.872489 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.872694 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.872776 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:19.875224 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:19.875226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:19.875227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:19.875236 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:19.876627 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:19.877156 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:19.877157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:19.877157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:19.877226 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:19.877516 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:19.877517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:19.877517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:19.877643 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:19.877705 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:19.877979 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:48:19.878076 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:19.878362 22831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:19.868985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:59655)
0811 18:48:19.869147 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:19.869193 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:19.869693 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:19.869909 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:19.869913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:19.870515 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:19.870729 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:19.870734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.871490 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.871491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:19.872385 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:19.872388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:19.872817 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:19.872819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:19.872949 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:19.873613 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:19.873624 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:19.875125 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:19.876714 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:19.876724 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:19.876727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:19.877051 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:19.877299 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:19.877300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:19.877300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:19.877440 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:19.877702 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:19.877710 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:19.877857 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 18:48:19.877949 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:48:19.878120 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260811 18:48:19.878898 22827 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37933
I20260811 18:48:19.878997 22827 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:19.879182 22827 heartbeater.cc:511] Master 127.9.215.62:37933 requested a full tablet report, sending...
I20260811 18:48:19.879593 22209 ts_manager.cc:194] Registered new tserver with Master: 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427)
I20260811 18:48:19.880113 22209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:59655
WARNING: no policy specified for oryx/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab.
Entry for principal oryx/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab.
I20260811 18:48:19.905645 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37933
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:19.994997 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:19.995249 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:19.995275 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:19.996650 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:19.996712 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:19.996742 22837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:19.998538 22837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:19.998809 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:19.999020 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:20.004602 22852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.004594 22855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.004622 22850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.004828 22837 server_base.cc:1034] running on GCE node
I20260811 18:48:20.005147 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:20.005401 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:20.006577 22837 hybrid_clock.cc:648] HybridClock initialized: now 1786474100006563 us; error 31 us; skew 500 ppm
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:20.009531 22837 init.cc:377] Logged in from keytab as oryx/127.9.215.3@KRBTEST.COM (short username oryx)
I20260811 18:48:20.010051 22837 webserver.cc:460] Webserver started at http://127.9.215.3:39391/ using document root <none> and password file <none>
I20260811 18:48:20.010281 22837 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.010334 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.010461 22837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:20.011582 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "30511577f0b74d5094e9f9381c4fc97c"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7ee2efef3179bbfcb24bb67d891cf328"
server_key_iv: "f827dc0151af0ad21d926f76acedf6b5"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.011945 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "30511577f0b74d5094e9f9381c4fc97c"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7ee2efef3179bbfcb24bb67d891cf328"
server_key_iv: "f827dc0151af0ad21d926f76acedf6b5"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.013224 22837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:20.014007 22865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.014163 22837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:20.014251 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "30511577f0b74d5094e9f9381c4fc97c"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7ee2efef3179bbfcb24bb67d891cf328"
server_key_iv: "f827dc0151af0ad21d926f76acedf6b5"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.014312 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.022842 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:20.025488 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:20.025651 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:20.025899 22837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:20.026230 22837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:20.026268 22837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.026299 22837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:20.026328 22837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.032390 22837 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:43959
I20260811 18:48:20.032447 23082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:43959 every 8 connection(s)
I20260811 18:48:20.032802 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:20.042235 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22837
I20260811 18:48:20.042325 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:20.042503 10076 external_mini_cluster.cc:1428] Setting key 54c8c5c51b5391d698619c57a336d902
I20260811 18:48:20.043543 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.033698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:52241 (local address 127.9.215.62:37933)
0811 18:48:20.033758 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.033760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.034041 (+ 281us) server_negotiation.cc:407] Connection header received
0811 18:48:20.034642 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.034644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.034656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.034715 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.035863 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.036413 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.037552 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.037772 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.037829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.040300 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.040301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.040302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.040310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.041792 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.042433 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.042434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.042434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.042503 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.042778 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.042779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.042779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.042932 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.042994 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.043331 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.043425 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:48:20.043777 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.033705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:52241)
0811 18:48:20.033902 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.033946 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.034521 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.034830 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.034834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.035488 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.035753 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.035759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.036545 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.036546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.037470 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.037473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.037845 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.037846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.037957 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.038520 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.038530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.040152 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.041898 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.041908 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.041911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.042321 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.042567 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.042568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.042568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.042713 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.042992 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.043002 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.043182 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.043311 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.043542 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260811 18:48:20.044183 23085 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37933
I20260811 18:48:20.044279 23085 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:20.044449 23085 heartbeater.cc:511] Master 127.9.215.62:37933 requested a full tablet report, sending...
I20260811 18:48:20.044799 22209 ts_manager.cc:194] Registered new tserver with Master: 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:20.045150 22209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:52241
I20260811 18:48:20.054395 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.215.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.61.keytab.
Entry for principal oryx/127.9.215.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.61.keytab.
I20260811 18:48:20.084640 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:35303
--webserver_interface=127.9.215.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.61.keytab
--principal=oryx/127.9.215.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37933,127.9.215.61:35303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:20.171115 23093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:20.171365 23093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:20.171401 23093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:20.172735 23093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:20.172801 23093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:20.172821 23093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:20.172837 23093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:20.172854 23093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:20.174538 23093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37933,127.9.215.61:35303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.61.keytab
--principal=oryx/127.9.215.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.215.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
W20260811 18:48:20.174695 23093 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.215.62:37933,127.9.215.61:35303'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 18:48:20.174867 23093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:20.175092 23093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:20.180441 23107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.180446 23106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.180449 23110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.180794 23093 server_base.cc:1034] running on GCE node
I20260811 18:48:20.181057 23093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:20.181409 23093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:20.182559 23093 hybrid_clock.cc:648] HybridClock initialized: now 1786474100182539 us; error 34 us; skew 500 ppm
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:20.185240 23093 init.cc:377] Logged in from keytab as oryx/127.9.215.61@KRBTEST.COM (short username oryx)
I20260811 18:48:20.185740 23093 webserver.cc:460] Webserver started at http://127.9.215.61:43767/ using document root <none> and password file <none>
I20260811 18:48:20.185956 23093 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.186008 23093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.186143 23093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:20.187252 23093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/instance:
uuid: "cfb18b42f7b545f185f22a46bcf9267d"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7bc855b8b1e3fabeed55d95e1eec9970"
server_key_iv: "10d5d3ccdb71e9ed4599bf9ee28d925c"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.187633 23093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal/instance:
uuid: "cfb18b42f7b545f185f22a46bcf9267d"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7bc855b8b1e3fabeed55d95e1eec9970"
server_key_iv: "10d5d3ccdb71e9ed4599bf9ee28d925c"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.189086 23093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:20.190008 23122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.190250 23093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:20.190429 23093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
uuid: "cfb18b42f7b545f185f22a46bcf9267d"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7bc855b8b1e3fabeed55d95e1eec9970"
server_key_iv: "10d5d3ccdb71e9ed4599bf9ee28d925c"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.190536 23093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.201098 23093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:20.203831 23093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:20.203974 23093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:20.208829 23093 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.61:35303
I20260811 18:48:20.208868 23220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.61:35303 every 8 connection(s)
I20260811 18:48:20.209273 23093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
I20260811 18:48:20.210429 23221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:20.210899 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 23093
I20260811 18:48:20.212371 23221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.61@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
I20260811 18:48:20.221052 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.213087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38814 (local address 127.9.215.62:37933)
0811 18:48:20.213227 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.213230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.213270 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:48:20.213352 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.213353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.213364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.213411 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.213906 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.214427 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.215257 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.215475 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.215543 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.217785 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.217786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.217787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.217795 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.219354 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.219867 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.219868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.219869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.219933 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.220238 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.220239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.220239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.220442 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.220513 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.220855 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.220955 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 18:48:20.221295 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.211626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.9.215.61:35303)
0811 18:48:20.211828 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.211837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.211865 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:48:20.212333 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.212341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.212406 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.212558 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.213229 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.214118 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.214864 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.215269 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.215368 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.218326 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.218336 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.218336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.218343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.219974 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.220355 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.220359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.220359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.220416 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.220607 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.220608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.220609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.220804 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.220969 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.221045 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.221110 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260811 18:48:20.221295 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.213001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:38814)
0811 18:48:20.213162 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.213203 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.213288 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.213430 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.213433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.213698 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.213809 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.213814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.214526 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.214527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.215168 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.215170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.215556 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.215557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.215623 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.216124 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.216133 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.217660 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.219438 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.219443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.219446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.219767 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.220000 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.220001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.220001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.220134 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.220524 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.220530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.220704 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.220825 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.221024 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260811 18:48:20.222075 23221 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.61@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
I20260811 18:48:20.228840 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.222356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:60608)
0811 18:48:20.222396 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.222408 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.222457 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.222564 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.222565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.222690 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.222747 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.222748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.223442 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.223443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.223994 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.223996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.224383 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.224384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.224451 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.224851 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.224855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.226103 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.227785 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.227787 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.227787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.227955 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.228168 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.228169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.228169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.228235 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.228542 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.228543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.228591 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.228652 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.228763 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 18:48:20.228919 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.222403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60608 (local address 127.9.215.61:35303)
0811 18:48:20.222439 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.222440 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.222447 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:20.222510 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.222511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.222519 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.222607 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.222831 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.223344 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.224082 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.224301 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.224358 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.226206 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.226208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.226208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.226214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.227666 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.228019 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.228020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.228021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.228072 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.228320 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.228322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.228322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.228450 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.228515 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.228694 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.228817 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":19}
I20260811 18:48:20.230533 23221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Bootstrap starting.
I20260811 18:48:20.231418 23221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:20.231778 23221 log.cc:824] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:20.232681 23221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: No bootstrap required, opened a new log
I20260811 18:48:20.234160 23221 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } }
I20260811 18:48:20.234313 23221 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:20.234337 23221 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb18b42f7b545f185f22a46bcf9267d, State: Initialized, Role: FOLLOWER
I20260811 18:48:20.234465 23221 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } }
I20260811 18:48:20.234867 23221 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:20.234851 23233 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } } }
I20260811 18:48:20.234946 23233 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:48:20.236685 23254 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:48:20.236794 23254 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:48:20.238752 23093 master_runner.cc:428] Detected that this master cfb18b42f7b545f185f22a46bcf9267d is joining an existing cluster
I20260811 18:48:20.238857 23093 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.215.61:35303
I20260811 18:48:20.246315 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.240028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:60620)
0811 18:48:20.240200 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.240212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.240297 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.240468 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.240469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.240761 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.240866 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.240868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.241512 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.241513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.242269 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.242271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.242748 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.242748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.242805 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.243310 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.243314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.243735 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.245296 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.245297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.245297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.245477 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.245671 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.245672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.245672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.245722 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.245944 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.245945 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.245987 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.246048 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.246181 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":5,"thread_start_us":110,"threads_started":1}
I20260811 18:48:20.246318 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.240092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60620 (local address 127.9.215.61:35303)
0811 18:48:20.240138 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.240140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.240273 (+ 133us) server_negotiation.cc:407] Connection header received
0811 18:48:20.240363 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.240364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.240374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.240425 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.240949 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.241449 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.242351 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.242628 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.242662 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.243816 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.243817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.243817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.243825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.245225 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.245554 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.245556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.245556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.245610 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.245779 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.245780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.245780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.245888 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.245948 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.246075 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.246244 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":21}
I20260811 18:48:20.247792 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.240113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.9.215.62:37933)
0811 18:48:20.240171 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.240173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.240391 (+ 218us) server_negotiation.cc:407] Connection header received
0811 18:48:20.240457 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.240458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.240470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.240519 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.240947 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.241757 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.242752 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.243447 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.243544 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.244678 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.244680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.244681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.244689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.246272 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.246877 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.246878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.246878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.246945 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.247243 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.247244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.247245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.247376 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.247445 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.247639 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.247725 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:48:20.247846 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.240028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:38828)
0811 18:48:20.240323 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.240333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.240393 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.240527 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.240528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.240761 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.240866 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.240868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.241852 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.241853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.242668 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.242670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.243608 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.243609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.243683 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.244115 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.244127 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.244566 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.246366 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.246367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.246367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.246684 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.247071 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.247071 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.247072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.247146 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.247449 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.247450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.247538 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.247611 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.247726 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":156,"thread_start_us":79,"threads_started":1}
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.62@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
I20260811 18:48:20.259374 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.250711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.9.215.61:35303)
0811 18:48:20.250763 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.250765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.250999 (+ 234us) server_negotiation.cc:407] Connection header received
0811 18:48:20.251041 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.251042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.251053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.251093 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.251772 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.252260 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.253630 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.253868 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.253934 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.256578 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.256580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.256581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.256611 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.257986 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.258507 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.258508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.258508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.258568 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.258816 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.258817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.258817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.258952 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.259016 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.259219 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.259299 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:48:20.259506 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.250611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:60630)
0811 18:48:20.250787 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.250845 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.250945 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.251131 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.251137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.251588 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.251673 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.251676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.252340 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.252341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.253509 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.253516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.253949 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.253950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.254062 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.254650 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.254656 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.256452 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.258077 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.258088 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.258091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.258426 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.258660 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.258661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.258661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.258734 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.259017 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.259026 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.259091 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.259199 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.259347 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260811 18:48:20.260231 22209 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.215.61:35303: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d"
member_type: NON_VOTER
last_known_addr {
host: "127.9.215.61"
port: 35303
}
attrs {
promote: true
}
}
dest_uuid: "5abd1ef3b2954060a1020004d81f4bf1"
cas_config_opid_index: -1
I20260811 18:48:20.260667 22209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } }
I20260811 18:48:20.261193 23273 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } }
I20260811 18:48:20.261514 23171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:20.262439 23273 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.215.61:35303
I20260811 18:48:20.262478 23171 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Refusing update from remote peer 5abd1ef3b2954060a1020004d81f4bf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 18:48:20.262945 23268 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:20.263082 23093 master.cc:537] Master@127.9.215.61:35303 shutting down...
I20260811 18:48:20.263268 23274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } }
I20260811 18:48:20.263566 23274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
W20260811 18:48:20.263963 22184 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 18:48:20.264461 22184 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 -> Peer cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61:35303): Couldn't send request to peer cfb18b42f7b545f185f22a46bcf9267d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:48:20.266383 23093 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:20.266498 23093 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:20.266556 23093 tablet_replica.cc:330] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: stopping tablet replica
I20260811 18:48:20.279840 23093 master.cc:552] Master@127.9.215.61:35303 shutdown complete.
I20260811 18:48:20.279927 23093 master_runner.cc:305] Clearing existing system tablet
I20260811 18:48:20.280067 23093 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.280210 23093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.281229 23093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:20.281807 23282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.281991 23093 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.282053 23093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.282078 23093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
uuid: "cfb18b42f7b545f185f22a46bcf9267d"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7bc855b8b1e3fabeed55d95e1eec9970"
server_key_iv: "10d5d3ccdb71e9ed4599bf9ee28d925c"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.282903 23093 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:20.284464 23093 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:48:20.284541 23093 log.cc:1196] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 18:48:20.284789 23093 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Deleting consensus metadata
I20260811 18:48:20.284904 23093 master_runner.cc:315] Copying system tablet from 127.9.215.62:37933
I20260811 18:48:20.285542 23093 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: tablet copy: Beginning tablet copy session from remote peer at address 127.9.215.62:37933
I20260811 18:48:20.292725 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.285893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:38838)
0811 18:48:20.286015 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.286026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.286088 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.286254 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.286255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.286420 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.286478 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.286480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.287525 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.287526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.288153 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.288156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.288692 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.288693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.288829 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.289339 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.289343 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.289672 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.291548 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.291549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.291549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.291752 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.291990 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.291991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.291991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.292064 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.292395 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.292396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.292457 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.292523 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.292633 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":77,"threads_started":1}
I20260811 18:48:20.292754 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.285976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.9.215.62:37933)
0811 18:48:20.286034 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.286036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.286081 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:48:20.286144 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.286145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.286156 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.286250 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.286545 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.287343 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.288247 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.288568 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.288651 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.289764 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.289765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.289766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.289773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.291381 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.291826 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.291827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.291828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.291899 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.292130 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.292131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.292131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.292306 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.292380 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.292560 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.292655 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:48:20.293138 22248 tablet_copy_service.cc:140] P 5abd1ef3b2954060a1020004d81f4bf1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cfb18b42f7b545f185f22a46bcf9267d ({username='oryx', principal='oryx/127.9.215.61@KRBTEST.COM'} at 127.0.0.1:38838)
I20260811 18:48:20.293356 22248 tablet_copy_service.cc:161] P 5abd1ef3b2954060a1020004d81f4bf1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cfb18b42f7b545f185f22a46bcf9267d at {username='oryx', principal='oryx/127.9.215.61@KRBTEST.COM'} at 127.0.0.1:38838: session id = cfb18b42f7b545f185f22a46bcf9267d-00000000000000000000000000000000
I20260811 18:48:20.294162 22248 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:48:20.295080 23093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:48:20.296375 23093 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: tablet copy: Starting download of 0 data blocks...
I20260811 18:48:20.296521 23093 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: tablet copy: Starting download of 1 WAL segments...
I20260811 18:48:20.297851 23093 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:48:20.299065 22248 tablet_copy_service.cc:342] P 5abd1ef3b2954060a1020004d81f4bf1: Request end of tablet copy session cfb18b42f7b545f185f22a46bcf9267d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.215.61@KRBTEST.COM'} at 127.0.0.1:38838
I20260811 18:48:20.299161 22248 tablet_copy_service.cc:433] P 5abd1ef3b2954060a1020004d81f4bf1: ending tablet copy session cfb18b42f7b545f185f22a46bcf9267d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cfb18b42f7b545f185f22a46bcf9267d
I20260811 18:48:20.301898 23093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:20.303324 23300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.303261 23308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.303541 23093 server_base.cc:1034] running on GCE node
W20260811 18:48:20.303427 23302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.303776 23093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:20.303918 23093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:20.305069 23093 hybrid_clock.cc:648] HybridClock initialized: now 1786474100305050 us; error 35 us; skew 500 ppm
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:20.306669 23093 init.cc:377] Logged in from keytab as oryx/127.9.215.61@KRBTEST.COM (short username oryx)
I20260811 18:48:20.306982 23093 webserver.cc:460] Webserver started at http://127.9.215.61:32987/ using document root <none> and password file <none>
I20260811 18:48:20.307108 23093 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.307145 23093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.307938 23093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:20.308580 23323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.308722 23093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:20.308763 23093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
uuid: "cfb18b42f7b545f185f22a46bcf9267d"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7bc855b8b1e3fabeed55d95e1eec9970"
server_key_iv: "10d5d3ccdb71e9ed4599bf9ee28d925c"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.308828 23093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.320310 23093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:20.322096 23093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:20.325253 23093 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.61:35303
I20260811 18:48:20.325495 23446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.61:35303 every 8 connection(s)
I20260811 18:48:20.325649 23093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
I20260811 18:48:20.326797 23447 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:48:20.327500 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Bootstrap starting.
I20260811 18:48:20.331023 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:48:20.331529 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: Bootstrap complete.
I20260811 18:48:20.331820 23447 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } }
I20260811 18:48:20.331934 23447 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cfb18b42f7b545f185f22a46bcf9267d, State: Initialized, Role: LEARNER
I20260811 18:48:20.331981 23447 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } }
I20260811 18:48:20.332202 23447 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:20.332213 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } }
I20260811 18:48:20.332347 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: LEARNER
I20260811 18:48:20.333860 23468 catalog_manager.cc:1259] Loaded cluster ID: b6358a463d964b94bea1b908646655ad
I20260811 18:48:20.334156 23468 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: loading cluster ID for follower catalog manager: success
I20260811 18:48:20.335254 23468 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: acquiring CA information for follower catalog manager: success
I20260811 18:48:20.335628 23468 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:48:20.380138 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.373764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.9.215.61:35303)
0811 18:48:20.373919 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.373921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.373928 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:20.373953 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.373953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.373963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.374017 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.374680 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.375365 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.376124 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.376341 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.376399 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.377564 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.377566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.377566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.377575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.378997 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.379325 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.379326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.379326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.379381 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.379581 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.379582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.379583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.379701 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.379799 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.379901 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.379990 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.9.215.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.60.keytab.
Entry for principal oryx/127.9.215.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.60.keytab.
I20260811 18:48:20.412046 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:39169
--webserver_interface=127.9.215.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.60.keytab
--principal=oryx/127.9.215.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:20.507786 23478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:20.508065 23478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:20.508101 23478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:20.509627 23478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:20.509689 23478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:20.509709 23478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:20.509724 23478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:20.509739 23478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:20.511623 23478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.60.keytab
--principal=oryx/127.9.215.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:39169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.215.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:20.511936 23478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:20.512183 23478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:20.517268 23486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.517282 23490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.517480 23488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.517932 23478 server_base.cc:1034] running on GCE node
I20260811 18:48:20.518209 23478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:20.518564 23478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:20.519735 23478 hybrid_clock.cc:648] HybridClock initialized: now 1786474100519722 us; error 33 us; skew 500 ppm
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:20.522524 23478 init.cc:377] Logged in from keytab as oryx/127.9.215.60@KRBTEST.COM (short username oryx)
I20260811 18:48:20.523196 23478 webserver.cc:460] Webserver started at http://127.9.215.60:37011/ using document root <none> and password file <none>
I20260811 18:48:20.523486 23478 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.523554 23478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.523701 23478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:20.524824 23478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/instance:
uuid: "915c1c7f244d4ebeaf057569e2162b01"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "a319264cd9d794aadef68e8a332a856d"
server_key_iv: "c8117e34e8ebaec1fa1ac1a23a58ddbe"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.525195 23478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal/instance:
uuid: "915c1c7f244d4ebeaf057569e2162b01"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "a319264cd9d794aadef68e8a332a856d"
server_key_iv: "c8117e34e8ebaec1fa1ac1a23a58ddbe"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.526734 23478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:20.527767 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.528162 23478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:20.528290 23478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
uuid: "915c1c7f244d4ebeaf057569e2162b01"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "a319264cd9d794aadef68e8a332a856d"
server_key_iv: "c8117e34e8ebaec1fa1ac1a23a58ddbe"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.528391 23478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.545032 23478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:20.547511 23478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:20.547674 23478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:20.557475 23478 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.60:39169
I20260811 18:48:20.557595 23604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.60:39169 every 8 connection(s)
I20260811 18:48:20.558022 23478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
I20260811 18:48:20.558943 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 23478
I20260811 18:48:20.559142 23605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:20.561931 23605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.215.60@KRBTEST.COM
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.60@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:20.572654 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.562902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.9.215.62:37933)
0811 18:48:20.562956 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.562958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.563121 (+ 163us) server_negotiation.cc:407] Connection header received
0811 18:48:20.563286 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.563288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.563301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.563374 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.563918 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.564655 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.565912 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.566233 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.566318 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.569343 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.569345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.569345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.569352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.570808 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.571261 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.571262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.571262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.571324 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.571965 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.571966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.571967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.572123 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.572206 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.572459 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.572563 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:48:20.573019 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.559677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.9.215.60:39169)
0811 18:48:20.559847 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.559858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.559891 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:48:20.560357 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.560367 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.560436 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.560606 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.561436 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.563193 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.564319 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.564927 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.565172 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.568020 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.568029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.568029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.568037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.570567 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.571014 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.571017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.571018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.571118 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.571559 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.571560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.571561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.572132 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.572309 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.572407 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.572619 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260811 18:48:20.573308 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.562760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:38854)
0811 18:48:20.562933 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.562983 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.563153 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.563406 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.563408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.563755 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.563831 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.563833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.564765 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.564766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.565812 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.565816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.566419 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.566421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.566486 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.567130 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.567140 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.569209 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.570904 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.570911 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.570913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.571175 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.571538 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.571541 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.571542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.571827 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.572209 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.572215 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.572299 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.572404 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.573133 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":7,"thread_start_us":103,"threads_started":1}
I20260811 18:48:20.574522 23605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.60@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
I20260811 18:48:20.583640 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.574928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:60662)
0811 18:48:20.574986 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.574999 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.575044 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.575302 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.575303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.575503 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.575692 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.575695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.576597 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.576598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.577186 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.577188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.577726 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.577727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.577946 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.578528 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.578533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.580325 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.582369 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.582370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.582371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.582569 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.582824 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.582825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.582825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.582884 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.583266 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.583267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.583322 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.583385 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.583558 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 18:48:20.583691 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.575045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.9.215.61:35303)
0811 18:48:20.575146 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.575148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.575156 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:20.575191 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.575191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.575204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.575271 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.575847 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.576494 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.577288 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.577520 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.577578 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.580459 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.580461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.580461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.580471 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.582246 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.582667 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.582668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.582669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.582735 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.582987 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.582988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.582989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.583154 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.583218 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.583520 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.583615 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:48:20.584285 23605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.60@KRBTEST.COM for oryx/127.9.215.60@KRBTEST.COM
I20260811 18:48:20.592052 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.584658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.9.215.60:39169)
0811 18:48:20.584773 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.584775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.584871 (+ 96us) server_negotiation.cc:407] Connection header received
0811 18:48:20.584949 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.584950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.584961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.585027 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.585448 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.586484 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.587202 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.587527 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.587588 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.589597 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.589598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.589599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.589606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.590966 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.591290 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.591291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.591292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.591340 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.591521 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.591522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.591523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.591638 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.591700 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.591843 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.591937 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:48:20.592113 23609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.584627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:40768)
0811 18:48:20.584767 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.584788 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.584893 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.585152 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.585153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.585294 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.585356 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.585358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.586596 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.586597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.587085 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.587087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.587618 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.587619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.587663 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.588063 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.588067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.589506 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.591060 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.591061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.591061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.591221 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.591394 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.591394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.591395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.591448 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.591715 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.591716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.591756 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.591816 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.591999 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":83}
I20260811 18:48:20.594116 23605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Bootstrap starting.
I20260811 18:48:20.595820 23605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:20.596516 23605 log.cc:824] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:20.597663 23605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: No bootstrap required, opened a new log
I20260811 18:48:20.601366 23605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } }
I20260811 18:48:20.601639 23605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:20.601673 23605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915c1c7f244d4ebeaf057569e2162b01, State: Initialized, Role: FOLLOWER
I20260811 18:48:20.601831 23605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } }
I20260811 18:48:20.602300 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } } }
I20260811 18:48:20.602380 23605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:20.602438 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:48:20.603070 23613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:20.603173 23613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } }
I20260811 18:48:20.603770 23613 leader_election.cc:288] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933), cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61:35303)
I20260811 18:48:20.603857 22234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "915c1c7f244d4ebeaf057569e2162b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5abd1ef3b2954060a1020004d81f4bf1" is_pre_election: true
I20260811 18:48:20.603842 23387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "915c1c7f244d4ebeaf057569e2162b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfb18b42f7b545f185f22a46bcf9267d" is_pre_election: true
I20260811 18:48:20.603991 23387 raft_consensus.cc:1799] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Handling vote request from an unknown peer 915c1c7f244d4ebeaf057569e2162b01
I20260811 18:48:20.604034 23387 raft_consensus.cc:2396] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 915c1c7f244d4ebeaf057569e2162b01 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260811 18:48:20.604598 23513 leader_election.cc:302] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915c1c7f244d4ebeaf057569e2162b01; no voters: 5abd1ef3b2954060a1020004d81f4bf1, cfb18b42f7b545f185f22a46bcf9267d
I20260811 18:48:20.604818 23613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:20.606396 23613 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 18:48:20.608067 23629 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:48:20.608145 23629 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:48:20.610286 23478 master_runner.cc:428] Detected that this master 915c1c7f244d4ebeaf057569e2162b01 is joining an existing cluster
I20260811 18:48:20.610342 23478 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.215.60:39169
I20260811 18:48:20.625694 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22326
I20260811 18:48:20.697685 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.612314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:60674)
0811 18:48:20.612854 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.612865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.612928 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.613107 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.613108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.613595 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.613824 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.613826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.614771 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.614772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.616279 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.616282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.616809 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.616973 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.693933 (+ 76960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.693940 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.694452 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.696419 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.696421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.696421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.696640 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.696891 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.696892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.696892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.696948 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.697271 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.697272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.697324 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.697380 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.697496 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":27,"thread_start_us":327,"threads_started":1}
I20260811 18:48:20.698211 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.612155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.9.215.61:35303)
0811 18:48:20.612200 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.612202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.612919 (+ 717us) server_negotiation.cc:407] Connection header received
0811 18:48:20.612983 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.612984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.612996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.613054 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.613908 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.614688 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616380 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.616700 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616756 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.694618 (+ 77862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.694620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.694620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.694630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.696277 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.696721 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.696722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.696723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.696784 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.697037 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.697038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.697039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.697187 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.697244 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.698015 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.698124 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:20.699527 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:43179
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:37933
--webserver_port=41057
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:48:20.701818 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.612527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.9.215.60:39169)
0811 18:48:20.612610 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.612612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.613127 (+ 515us) server_negotiation.cc:407] Connection header received
0811 18:48:20.613205 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.613206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.613213 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.613251 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.613998 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.614702 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616025 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.616352 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616406 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.698629 (+ 82223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.698630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.698630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.698638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.700217 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.700541 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.700543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.700543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.700598 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.700747 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.700748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.700748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.700868 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.700905 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.701619 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.701706 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":31}
I20260811 18:48:20.703321 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.612617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:40772)
0811 18:48:20.613070 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.613078 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.613148 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.613306 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.613307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.613687 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.613903 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.613906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.614792 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.614793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.615929 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.615933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.616463 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.616597 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.617239 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.617245 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.698514 (+ 81269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.700309 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.700310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.700310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.700476 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.700654 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.700654 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.700654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.700696 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.700939 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.700939 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.700983 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.701945 (+ 962us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.702459 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":238,"threads_started":1}
I20260811 18:48:20.706058 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.612090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.9.215.62:37933)
0811 18:48:20.612162 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.612164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.612573 (+ 409us) server_negotiation.cc:407] Connection header received
0811 18:48:20.612678 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.612680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.612693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.612758 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.613687 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.614494 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616394 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.616700 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.616743 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.701133 (+ 84390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.701135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.701135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.701145 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.703253 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.704032 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.704034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.704034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.704121 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.705275 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.705277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.705277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.705507 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.705583 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.705858 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.705970 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:48:20.706516 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.612069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:38858)
0811 18:48:20.612500 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.612513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.612605 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.612780 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.612781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.613291 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.613576 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.613580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.614586 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.614587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.616292 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.616294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.617229 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.617229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.617344 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.617819 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.617824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.699681 (+ 81857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.703693 (+ 4012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.703694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.703695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.703924 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.704228 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.704229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.704229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.705163 (+ 934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.705649 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.705650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.705701 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.706211 (+ 510us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.706343 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":37,"thread_start_us":244,"threads_started":1}
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474099, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.62@KRBTEST.COM for oryx/127.9.215.60@KRBTEST.COM
I20260811 18:48:20.716008 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.708930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:40786)
0811 18:48:20.708987 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.708997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.709039 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.709240 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.709241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.709412 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.709461 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.709464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.710183 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.710185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.710816 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.710820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.711157 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.711157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.711229 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.711707 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.711713 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.713131 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.714850 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.714851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.714851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.715032 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.715215 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.715216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.715216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.715277 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.715685 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.715686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.715745 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.715811 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.715930 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 18:48:20.716341 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.708979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.9.215.60:39169)
0811 18:48:20.709128 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.709130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.709138 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:20.709174 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.709175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.709185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.709233 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.709563 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.710106 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.710887 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.711095 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.711151 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.713231 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.713233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.713233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.713242 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.714752 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.715104 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.715105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.715105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.715159 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.715327 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.715327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.715328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.715443 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.715682 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.716172 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.716258 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:48:20.716889 22209 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.215.60:39169: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01"
member_type: NON_VOTER
last_known_addr {
host: "127.9.215.60"
port: 39169
}
attrs {
promote: true
}
}
dest_uuid: "5abd1ef3b2954060a1020004d81f4bf1"
cas_config_opid_index: 5
I20260811 18:48:20.717163 22209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } }
I20260811 18:48:20.717814 23274 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.718737 23274 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.215.60:39169
I20260811 18:48:20.718823 23274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.718901 23274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:20.719236 23478 master.cc:537] Master@127.9.215.60:39169 shutting down...
I20260811 18:48:20.719425 23559 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Refusing update from remote peer 5abd1ef3b2954060a1020004d81f4bf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:48:20.720336 23268 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:48:20.721315 22181 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 -> Peer 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60:39169): Couldn't send request to peer 915c1c7f244d4ebeaf057569e2162b01. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:48:20.723805 23478 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:20.723923 23478 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:20.723974 23478 tablet_replica.cc:330] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: stopping tablet replica
I20260811 18:48:20.726263 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.717676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:60682)
0811 18:48:20.722497 (+ 4821us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.722513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.722590 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.722912 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.722913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.722916 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:20.722972 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.722975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.723706 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.723708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.725060 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.725065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.725946 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.725948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.726020 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.726060 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.726126 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.726181 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4795}
I20260811 18:48:20.726661 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.717747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.9.215.61:35303)
0811 18:48:20.717791 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.717793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.722705 (+ 4912us) server_negotiation.cc:407] Connection header received
0811 18:48:20.722741 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.722742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.722771 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.722832 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:20.723049 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.723619 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.725176 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.725769 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.725830 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.725906 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.726537 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.726592 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:48:20.727165 23387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Refusing update from remote peer 5abd1ef3b2954060a1020004d81f4bf1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:48:20.727387 23268 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:48:20.728055 23478 master.cc:552] Master@127.9.215.60:39169 shutdown complete.
I20260811 18:48:20.728101 23478 master_runner.cc:305] Clearing existing system tablet
I20260811 18:48:20.728230 23478 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.728271 23478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.728170 23387 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.729423 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5abd1ef3b2954060a1020004d81f4bf1. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.729509 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: LEARNER
I20260811 18:48:20.729544 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: NON_VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: true } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.729580 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: LEARNER
I20260811 18:48:20.729678 23268 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: attempting to promote NON_VOTER cfb18b42f7b545f185f22a46bcf9267d to VOTER
I20260811 18:48:20.729924 23268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } }
I20260811 18:48:20.730022 23478 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:20.730647 23387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 LEARNER]: Refusing update from remote peer 5abd1ef3b2954060a1020004d81f4bf1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 18:48:20.730855 23273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 18:48:20.730947 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.730327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:40800)
0811 18:48:20.730862 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.730920 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515}
I20260811 18:48:20.731225 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.731150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:40804)
0811 18:48:20.731182 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.731202 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:39169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260811 18:48:20.731283 22181 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 -> Peer 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60:39169): Couldn't send request to peer 915c1c7f244d4ebeaf057569e2162b01. Status: Network error: Client connection negotiation failed: client connection to 127.9.215.60:39169: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:48:20.731377 23663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.731535 23478 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.731590 23478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:48:20.731627 23478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
uuid: "915c1c7f244d4ebeaf057569e2162b01"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "a319264cd9d794aadef68e8a332a856d"
server_key_iv: "c8117e34e8ebaec1fa1ac1a23a58ddbe"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.732008 23273 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.732774 23478 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:20.733348 23273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.733431 23273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:20.733484 23273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.733523 23273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:20.733964 23387 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.734365 23478 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:48:20.734431 23478 log.cc:1196] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 18:48:20.734640 23478 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Deleting consensus metadata
I20260811 18:48:20.734997 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.735088 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:48:20.735776 23478 master_runner.cc:315] Copying system tablet from 127.9.215.62:37933
I20260811 18:48:20.736567 23478 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: tablet copy: Beginning tablet copy session from remote peer at address 127.9.215.62:37933
I20260811 18:48:20.743211 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.736936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.9.215.62:37933)
0811 18:48:20.736984 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.736986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.737098 (+ 112us) server_negotiation.cc:407] Connection header received
0811 18:48:20.737163 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.737164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.737175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.737223 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.737704 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.738422 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.739326 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.739598 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.739659 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.740631 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.740632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.740633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.740641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.742118 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.742459 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.742461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.742461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.742519 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.742682 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.742683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.742683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.742809 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.742868 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.743014 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.743104 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:48:20.743228 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.736897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:38874)
0811 18:48:20.737033 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.737043 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.737104 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.737229 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.737230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.737531 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.737598 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.737601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.738530 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.738531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.739216 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.739224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.739660 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.739662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.739728 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.740208 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.740213 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.740542 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.742206 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.742208 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.742208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.742393 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.742572 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.742573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.742573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.742632 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.742866 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.742867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.742919 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.742979 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.743114 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260811 18:48:20.743929 22248 tablet_copy_service.cc:140] P 5abd1ef3b2954060a1020004d81f4bf1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 915c1c7f244d4ebeaf057569e2162b01 ({username='oryx', principal='oryx/127.9.215.60@KRBTEST.COM'} at 127.0.0.1:38874)
I20260811 18:48:20.743999 22248 tablet_copy_service.cc:161] P 5abd1ef3b2954060a1020004d81f4bf1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 915c1c7f244d4ebeaf057569e2162b01 at {username='oryx', principal='oryx/127.9.215.60@KRBTEST.COM'} at 127.0.0.1:38874: session id = 915c1c7f244d4ebeaf057569e2162b01-00000000000000000000000000000000
I20260811 18:48:20.744436 22248 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:48:20.744966 23478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:48:20.745990 23478 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: tablet copy: Starting download of 0 data blocks...
I20260811 18:48:20.746148 23478 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: tablet copy: Starting download of 1 WAL segments...
I20260811 18:48:20.746943 23478 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:48:20.747844 22248 tablet_copy_service.cc:342] P 5abd1ef3b2954060a1020004d81f4bf1: Request end of tablet copy session 915c1c7f244d4ebeaf057569e2162b01-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.215.60@KRBTEST.COM'} at 127.0.0.1:38874
I20260811 18:48:20.747905 22248 tablet_copy_service.cc:433] P 5abd1ef3b2954060a1020004d81f4bf1: ending tablet copy session 915c1c7f244d4ebeaf057569e2162b01-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 915c1c7f244d4ebeaf057569e2162b01
I20260811 18:48:20.750538 23478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:20.752105 23675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.752110 23678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.752357 23478 server_base.cc:1034] running on GCE node
W20260811 18:48:20.752638 23676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.752851 23478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:20.753012 23478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:20.754187 23478 hybrid_clock.cc:648] HybridClock initialized: now 1786474100754179 us; error 26 us; skew 500 ppm
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:20.755734 23478 init.cc:377] Logged in from keytab as oryx/127.9.215.60@KRBTEST.COM (short username oryx)
I20260811 18:48:20.756028 23478 webserver.cc:460] Webserver started at http://127.9.215.60:35835/ using document root <none> and password file <none>
I20260811 18:48:20.756158 23478 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.756193 23478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.756870 23478 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:20.757426 23690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.757552 23478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.757601 23478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
uuid: "915c1c7f244d4ebeaf057569e2162b01"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "a319264cd9d794aadef68e8a332a856d"
server_key_iv: "c8117e34e8ebaec1fa1ac1a23a58ddbe"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.757668 23478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.811178 23478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:20.812877 23478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:20.815960 23478 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.60:39169
I20260811 18:48:20.815974 23777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.60:39169 every 8 connection(s)
I20260811 18:48:20.816323 23478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
I20260811 18:48:20.817193 23779 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:48:20.817754 23779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Bootstrap starting.
I20260811 18:48:20.820320 23779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:48:20.820547 23779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Bootstrap complete.
I20260811 18:48:20.820911 23779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } }
I20260811 18:48:20.820991 23779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 915c1c7f244d4ebeaf057569e2162b01, State: Initialized, Role: LEARNER
I20260811 18:48:20.821031 23779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } }
I20260811 18:48:20.821240 23779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:20.823086 23799 catalog_manager.cc:1259] Loaded cluster ID: b6358a463d964b94bea1b908646655ad
I20260811 18:48:20.823177 23799 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: loading cluster ID for follower catalog manager: success
I20260811 18:48:20.823557 23786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:20.823768 23786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:48:20.824182 23799 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: acquiring CA information for follower catalog manager: success
I20260811 18:48:20.824450 23799 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:48:20.834980 23654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:20.835342 23654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:20.835391 23654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:20.837632 23654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:20.837754 23654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:20.837808 23654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:20.840819 23654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:43179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=41057
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:20.841125 23654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:20.841427 23654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:20.847723 23808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.847728 23816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:20.847779 23811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:20.848343 23654 server_base.cc:1034] running on GCE node
I20260811 18:48:20.848588 23654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:20.848853 23654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:20.850023 23654 hybrid_clock.cc:648] HybridClock initialized: now 1786474100850007 us; error 32 us; skew 500 ppm
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:20.853061 23654 init.cc:377] Logged in from keytab as oryx/127.9.215.1@KRBTEST.COM (short username oryx)
I20260811 18:48:20.853688 23654 webserver.cc:460] Webserver started at http://127.9.215.1:41057/ using document root <none> and password file <none>
I20260811 18:48:20.854000 23654 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:20.854094 23654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:20.856142 23654 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 18:48:20.856968 23837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.857151 23654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:20.857259 23654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "3948537b697d4d77a7946977d83a5ac1"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "0230e43f9b3104c83ccc10d5c56a8bab"
server_key_iv: "f62089f8331ea893cde1f00e6bf0d571"
server_key_version: "encryptionkey@0"
I20260811 18:48:20.857811 23654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:20.872612 23654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:20.875656 23654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:20.875901 23654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:20.876278 23654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:20.876657 23654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:20.876708 23654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.876739 23654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:20.876756 23654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:20.881091 22827 heartbeater.cc:503] Master 127.9.215.62:37933 was elected leader, sending a full tablet report...
I20260811 18:48:20.891767 23654 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:43179
I20260811 18:48:20.891855 24064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:43179 every 8 connection(s)
I20260811 18:48:20.892233 23654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:20.898795 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 23654
I20260811 18:48:20.898963 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22582
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.60@KRBTEST.COM
Aug 11 18:48:20 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474100, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
I20260811 18:48:20.906471 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.893145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:47945 (local address 127.9.215.60:39169)
0811 18:48:20.893297 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.893300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.893597 (+ 297us) server_negotiation.cc:407] Connection header received
0811 18:48:20.894962 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.894963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.894978 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.895061 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.896886 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.897522 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.898587 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.899055 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.899125 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.902801 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.902802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.902803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.902812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.904547 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.905223 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.905225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.905225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.905293 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.905628 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.905629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.905629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.905794 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.905842 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.906254 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.906362 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260811 18:48:20.906764 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.893818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:58049 (local address 127.9.215.62:37933)
0811 18:48:20.893885 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.893886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.893914 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:48:20.894844 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.894845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.894859 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.894890 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.896009 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.896770 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.898343 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.898568 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.898645 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.902484 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.902485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.902486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.902495 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.904532 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.905241 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.905242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.905243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.905301 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.905628 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.905629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.905629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.906036 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.906081 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.906596 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.906683 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:48:20.907500 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.893149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.9.215.1:47945)
0811 18:48:20.893448 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.893523 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.894708 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.895083 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.895084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.895583 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.895878 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.895884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.897632 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.897634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.898469 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.898471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.899183 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.899184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.899310 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.899984 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.899988 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.902674 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.904636 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.904649 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.904652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.905127 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.905368 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.905368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.905369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.905548 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.905892 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.905899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.906089 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.906968 (+ 879us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.907137 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":6,"thread_start_us":182,"threads_started":1}
I20260811 18:48:20.907497 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.893571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:58049)
0811 18:48:20.893857 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.893868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.894721 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.895034 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.895040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.895592 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.895877 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.895884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.896983 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.896985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.898233 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.898235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.898721 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.898722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.898972 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.899863 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.899874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.902369 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.904636 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.904649 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.904652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.905147 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.905368 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.905368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.905369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.905548 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.906127 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.906128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.906239 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.906904 (+ 665us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.907202 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":124,"thread_start_us":122,"threads_started":1}
I20260811 18:48:20.908215 24066 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37933
I20260811 18:48:20.908332 24066 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:20.908558 24066 heartbeater.cc:511] Master 127.9.215.62:37933 requested a full tablet report, sending...
I20260811 18:48:20.908751 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.893275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:56377 (local address 127.9.215.61:35303)
0811 18:48:20.893330 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:20.893331 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:20.893599 (+ 268us) server_negotiation.cc:407] Connection header received
0811 18:48:20.894836 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:20.894838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:20.894851 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:20.894903 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:20.895981 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.896775 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.899323 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.899625 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.899677 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.905411 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:20.905413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:20.905414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:20.905424 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:20.907190 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.907710 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.907711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.907712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.907783 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:20.908014 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:20.908015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:20.908015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:20.908224 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:20.908289 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:20.908552 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 18:48:20.908647 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:48:20.909042 22209 ts_manager.cc:194] Re-registered known tserver with Master: 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179)
I20260811 18:48:20.909197 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:20.893149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.9.215.1:56377)
0811 18:48:20.893470 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:20.893501 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:20.894725 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:20.895043 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:20.895045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:20.895583 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:20.895877 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.895883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.898341 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.898341 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.899211 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:20.899214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:20.899759 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:20.899760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:20.899849 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:20.903053 (+ 3204us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:20.903058 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:20.905265 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:20.907303 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.907305 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.907305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.907613 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.907866 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:20.907867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:20.907868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:20.907936 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:20.908328 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:20.908329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:20.908376 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:48:20.908849 (+ 473us) client_negotiation.cc:240] Negotiation successful
0811 18:48:20.908983 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":3,"thread_start_us":175,"threads_started":1}
I20260811 18:48:20.909586 22209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:58049
I20260811 18:48:20.910084 24067 heartbeater.cc:347] Connected to a master server at 127.9.215.61:35303
I20260811 18:48:20.910162 24067 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:20.910342 24067 heartbeater.cc:511] Master 127.9.215.61:35303 requested a full tablet report, sending...
I20260811 18:48:20.910784 23364 ts_manager.cc:194] Registered new tserver with Master: 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179)
I20260811 18:48:20.939442 24065 heartbeater.cc:347] Connected to a master server at 127.9.215.60:39169
I20260811 18:48:20.939553 24065 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:20.939666 24065 heartbeater.cc:511] Master 127.9.215.60:39169 requested a full tablet report, sending...
I20260811 18:48:20.940040 23717 ts_manager.cc:194] Registered new tserver with Master: 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179)
I20260811 18:48:20.946076 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:33427
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:37933
--webserver_port=42011
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:21.036361 24078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:21.036592 24078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:21.036635 24078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:21.037955 24078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:21.038007 24078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:21.038048 24078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:21.039702 24078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:33427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=42011
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:21.039970 24078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:21.040205 24078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:21.046059 23085 heartbeater.cc:503] Master 127.9.215.62:37933 was elected leader, sending a full tablet report...
W20260811 18:48:21.046020 24086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:21.046020 24090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:21.046144 24085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:21.046617 24078 server_base.cc:1034] running on GCE node
I20260811 18:48:21.046967 24078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:21.047276 24078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:21.048444 24078 hybrid_clock.cc:648] HybridClock initialized: now 1786474101048416 us; error 87 us; skew 500 ppm
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:21.051363 24078 init.cc:377] Logged in from keytab as oryx/127.9.215.2@KRBTEST.COM (short username oryx)
I20260811 18:48:21.051949 24078 webserver.cc:460] Webserver started at http://127.9.215.2:42011/ using document root <none> and password file <none>
I20260811 18:48:21.052204 24078 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:21.052275 24078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:21.053807 24078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:48:21.054486 24104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:21.054663 24078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:21.054762 24078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "581b5d0e6de143b4a31bf7e2834108aa"
format_stamp: "Formatted at 2026-08-11 18:48:19 on dist-test-slave-pww0"
server_key: "42b59fa41f0042abac69656cebf8db96"
server_key_iv: "ced68e950a45508ae2211363824a1a58"
server_key_version: "encryptionkey@0"
I20260811 18:48:21.055137 24078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:21.072999 24078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:21.075287 24078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:21.075516 24078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:21.075784 24078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:21.076113 24078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:21.076165 24078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:21.076197 24078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:21.076221 24078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:21.082782 24078 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:33427
I20260811 18:48:21.082880 24336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:33427 every 8 connection(s)
I20260811 18:48:21.083218 24078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:21.092872 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 24078
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.60@KRBTEST.COM
I20260811 18:48:21.093021 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22837
I20260811 18:48:21.101583 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.084129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:44863 (local address 127.9.215.60:39169)
0811 18:48:21.084182 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.084184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.084661 (+ 477us) server_negotiation.cc:407] Connection header received
0811 18:48:21.085816 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.085817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.085830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.085881 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.086985 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.087805 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089160 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.089467 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089542 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.093482 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.093484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.093485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.093494 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.099904 (+ 6410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.100473 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.100475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.100475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.100530 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.100767 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.100767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.100767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.100897 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.100933 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.101285 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.101385 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:48:21.102308 24346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.084140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.9.215.2:44863)
0811 18:48:21.084601 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.084611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.085688 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.086017 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.086024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.086578 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:21.086913 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.086915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.087892 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.087892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.089025 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.089051 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.089557 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.089724 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.090492 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:21.090497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:21.093362 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:21.100034 (+ 6672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.100052 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.100055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.100394 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.100587 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.100587 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.100587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.100713 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.100971 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:21.100979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:21.101154 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.101776 (+ 622us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.102013 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":72,"thread_start_us":106,"threads_started":1}
I20260811 18:48:21.102768 24339 heartbeater.cc:347] Connected to a master server at 127.9.215.60:39169
I20260811 18:48:21.102861 24339 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:21.103049 24339 heartbeater.cc:511] Master 127.9.215.60:39169 requested a full tablet report, sending...
I20260811 18:48:21.103448 23717 ts_manager.cc:194] Registered new tserver with Master: 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427)
I20260811 18:48:21.105827 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.084131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:45089 (local address 127.9.215.61:35303)
0811 18:48:21.084214 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.084216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.084506 (+ 290us) server_negotiation.cc:407] Connection header received
0811 18:48:21.085795 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.085797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.085810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.085869 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.086981 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.087550 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089224 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.089512 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089571 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.093301 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.093304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.093304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.093314 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.103390 (+ 10076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.104884 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.104886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.104886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.104964 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.105233 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.105234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.105235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.105365 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.105402 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.105655 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.105745 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:48:21.106308 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.084140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.9.215.2:45089)
0811 18:48:21.084354 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.084395 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.085669 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.086017 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.086024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.086517 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:21.086843 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.086851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.087760 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.087761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.089094 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.089096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.089575 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.089724 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.090483 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:21.090495 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:21.093157 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:21.104494 (+ 11337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.104495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.104496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.104792 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.105108 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.105109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.105109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.105173 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.105437 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:21.105438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:21.105554 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.105902 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.106014 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":65,"spinlock_wait_cycles":8064,"thread_start_us":106,"threads_started":1}
I20260811 18:48:21.113251 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.084537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:41659 (local address 127.9.215.62:37933)
0811 18:48:21.084594 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.084596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.086082 (+ 1486us) server_negotiation.cc:407] Connection header received
0811 18:48:21.086135 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.086136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.086149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.086201 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.086944 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.087809 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089625 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.089817 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089874 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.107734 (+ 17860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.107736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.107737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.107746 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.111754 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.112269 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.112271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.112272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.112349 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.112594 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.112595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.112596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.112784 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.112835 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.113062 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.113155 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:48:21.113579 24342 heartbeater.cc:347] Connected to a master server at 127.9.215.61:35303
I20260811 18:48:21.113617 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.084456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:41659)
0811 18:48:21.086005 (+ 1549us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.086014 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.086077 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.086210 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.086210 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.086580 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:21.086844 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.086850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.088165 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.088165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.089089 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.089092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.089889 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.089890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.089983 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.090633 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:21.090639 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:21.107609 (+ 16970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:21.111886 (+ 4277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.111888 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.111888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.112162 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.112435 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.112435 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.112436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.112506 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.112888 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:21.112889 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:21.112947 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.113374 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.113502 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1503,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260811 18:48:21.113643 24342 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:21.113729 24342 heartbeater.cc:511] Master 127.9.215.61:35303 requested a full tablet report, sending...
I20260811 18:48:21.114032 24341 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37933
I20260811 18:48:21.114080 24341 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:21.114136 24341 heartbeater.cc:511] Master 127.9.215.62:37933 requested a full tablet report, sending...
I20260811 18:48:21.114423 22209 ts_manager.cc:194] Re-registered known tserver with Master: 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427)
I20260811 18:48:21.114806 22209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:41659
I20260811 18:48:21.115849 23364 ts_manager.cc:194] Registered new tserver with Master: 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427)
I20260811 18:48:21.117835 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:43959
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:37933
--webserver_port=39391
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:21.207789 24349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:21.208026 24349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:21.208055 24349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:21.209443 24349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:21.209501 24349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:21.209534 24349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:21.211302 24349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:43959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=39391
--tserver_master_addrs=127.9.215.62:37933,127.9.215.61:35303,127.9.215.60:39169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:21.211612 24349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:21.211889 24349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:21.217940 24349 server_base.cc:1034] running on GCE node
W20260811 18:48:21.217903 24356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:21.218031 24361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:21.217918 24365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:21.218289 24349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:21.218529 24349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:21.219682 24349 hybrid_clock.cc:648] HybridClock initialized: now 1786474101219664 us; error 34 us; skew 500 ppm
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:21.222532 24349 init.cc:377] Logged in from keytab as oryx/127.9.215.3@KRBTEST.COM (short username oryx)
I20260811 18:48:21.223160 24349 webserver.cc:460] Webserver started at http://127.9.215.3:39391/ using document root <none> and password file <none>
I20260811 18:48:21.223417 24349 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:21.223512 24349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:21.224954 24349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:21.225785 24375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:21.225951 24349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:21.226045 24349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "30511577f0b74d5094e9f9381c4fc97c"
format_stamp: "Formatted at 2026-08-11 18:48:20 on dist-test-slave-pww0"
server_key: "7ee2efef3179bbfcb24bb67d891cf328"
server_key_iv: "f827dc0151af0ad21d926f76acedf6b5"
server_key_version: "encryptionkey@0"
I20260811 18:48:21.226434 24349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:21.243960 24349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:21.246197 24349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:21.246346 24349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:21.246587 24349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:21.246904 24349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:21.246939 24349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:21.246970 24349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:21.246994 24349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:21.254081 24349 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:43959
I20260811 18:48:21.254132 24599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:43959 every 8 connection(s)
I20260811 18:48:21.254526 24349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:21.254664 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 24349
I20260811 18:48:21.260627 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.256073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.9.215.60:39169)
0811 18:48:21.256357 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.256359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.256364 (+ 5us) server_negotiation.cc:407] Connection header received
0811 18:48:21.256390 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.256391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.256418 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.256486 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:21.256675 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.258134 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.259530 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.260146 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.260196 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.260271 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.260498 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.260545 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":58,"threads_started":1}
I20260811 18:48:21.260892 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:40810)
0811 18:48:21.256055 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.256064 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.256118 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.256528 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.256528 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.256530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:21.256583 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.256585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.258236 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.258237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.259409 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.259413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.260341 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.260342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.260419 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.260424 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.260730 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.260805 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":50,"threads_started":1}
I20260811 18:48:21.261377 23740 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 18:48:21.261581 23786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5abd1ef3b2954060a1020004d81f4bf1. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:21.261699 23786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: This master's current role is: LEARNER
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.60@KRBTEST.COM
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.61@KRBTEST.COM
Aug 11 18:48:21 dist-test-slave-pww0 krb5kdc[22117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474101, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:21.267964 24613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.9.215.3:40531)
0811 18:48:21.256256 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.256266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.257247 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.257434 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.257435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.257626 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:21.257738 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.257745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.259407 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.259408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.260696 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.260699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.261169 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.261170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.261328 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.261937 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:21.261947 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:21.264145 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:21.266013 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.266024 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.266027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.266353 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.266572 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.266573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.266573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.266729 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.267047 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:21.267055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:21.267350 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.267482 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.267678 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":5,"thread_start_us":117,"threads_started":1}
I20260811 18:48:21.268520 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.256051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:40531 (local address 127.9.215.60:39169)
0811 18:48:21.256168 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.256170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.256315 (+ 145us) server_negotiation.cc:407] Connection header received
0811 18:48:21.257330 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.257332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.257342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.257398 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.258558 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.259280 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.260823 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.261093 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.261156 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.264255 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.264257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.264257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.264265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.265949 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.266437 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.266438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.266439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.266503 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.266817 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.266818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.266818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.266961 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.267019 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.268317 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.268414 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 18:48:21.268878 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:47581 (local address 127.9.215.61:35303)
0811 18:48:21.255996 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.255998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.256008 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:21.256993 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.256993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.257005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.257045 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.257831 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.258431 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.262045 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.262339 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.262383 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.265564 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.265566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.265567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.265575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.267116 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.267690 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.267691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.267691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.267781 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.268065 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.268066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.268066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.268281 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.268356 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.268635 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.268751 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:48:21.269331 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.9.215.3:47581)
0811 18:48:21.255574 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.255621 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.256897 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.257124 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.257128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.257489 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:21.257736 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.257749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.261208 (+ 3459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.261208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.261940 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.261944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.262462 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.262463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.262526 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.263166 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:21.263171 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:21.265031 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:21.267222 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.267223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.267224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.267570 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.267875 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.267876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.267876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.267960 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.268390 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:21.268391 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:21.268463 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.269042 (+ 579us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.269170 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":137,"threads_started":1}
I20260811 18:48:21.269378 24600 heartbeater.cc:347] Connected to a master server at 127.9.215.60:39169
I20260811 18:48:21.269502 24600 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:21.269682 24602 heartbeater.cc:347] Connected to a master server at 127.9.215.61:35303
I20260811 18:48:21.269726 24600 heartbeater.cc:511] Master 127.9.215.60:39169 requested a full tablet report, sending...
I20260811 18:48:21.269732 24602 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:21.270010 24602 heartbeater.cc:511] Master 127.9.215.61:35303 requested a full tablet report, sending...
I20260811 18:48:21.270320 23717 ts_manager.cc:194] Registered new tserver with Master: 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:21.270349 23364 ts_manager.cc:194] Registered new tserver with Master: 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:21.272671 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:41831 (local address 127.9.215.62:37933)
0811 18:48:21.255919 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.255921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.258496 (+ 2575us) server_negotiation.cc:407] Connection header received
0811 18:48:21.258577 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.258578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.258588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.258633 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.259013 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.259733 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.260937 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.261196 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.261248 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.266045 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.266046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.266047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.266054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.268044 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.271915 (+ 3871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.271916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.271917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.271992 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.272174 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.272175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.272175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.272292 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.272347 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.272500 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.272606 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:48:21.272914 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:41831)
0811 18:48:21.258423 (+ 2734us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.258434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.258521 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.258675 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.258676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.258888 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:21.258950 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.258952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.259797 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.259798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.260701 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.260704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.262359 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.262360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.262666 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.263845 (+ 1179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:21.263850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:21.265876 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:21.271553 (+ 5677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.271555 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.271555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.271808 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.272045 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:21.272046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:21.272046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:21.272113 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:21.272339 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:21.272340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:21.272403 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.272483 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.272605 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2718,"thread_start_us":97,"threads_started":1}
I20260811 18:48:21.273378 24601 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37933
I20260811 18:48:21.273438 24601 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:21.273515 24601 heartbeater.cc:511] Master 127.9.215.62:37933 requested a full tablet report, sending...
I20260811 18:48:21.273762 22209 ts_manager.cc:194] Re-registered known tserver with Master: 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:21.274222 22209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:41831
I20260811 18:48:21.275353 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.255958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.9.215.62:37933)
0811 18:48:21.256191 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.256192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.256374 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:48:21.256395 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.256396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.256407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.256469 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.257355 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.258143 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.260192 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.260506 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.260557 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.263648 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.263650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.263651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.263659 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.270595 (+ 6936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.273872 (+ 3277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.273874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.273874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.273946 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.274245 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.274246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.274246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.274390 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.274493 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.275151 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.275255 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":52,"threads_started":1}
I20260811 18:48:21.282442 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.256168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.9.215.60:39169)
0811 18:48:21.258253 (+ 2085us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.258255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.266962 (+ 8707us) server_negotiation.cc:407] Connection header received
0811 18:48:21.274127 (+ 7165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.274129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.274145 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.274186 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.275125 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.275867 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.276996 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.277294 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.277353 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.278774 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.278775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.278776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.278783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.280838 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.281399 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.281400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.281401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.281480 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.281795 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.281797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.281797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.281955 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.282058 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.282241 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.282343 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2066,"thread_start_us":61,"threads_started":1}
I20260811 18:48:21.283152 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.256597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60688 (local address 127.9.215.61:35303)
0811 18:48:21.258342 (+ 1745us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.258344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.259085 (+ 741us) server_negotiation.cc:407] Connection header received
0811 18:48:21.259168 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.259169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.259183 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.259237 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.261202 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.262281 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.266143 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.266485 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.266531 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.279688 (+ 13157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.279690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.279690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.279700 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.281506 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.282012 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.282013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.282014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.282084 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.282290 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.282291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.282291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.282460 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.282557 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.282937 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.283057 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1726,"thread_start_us":63,"threads_started":1}
I20260811 18:48:21.280966 22209 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:21.283653 22209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:21.284634 24635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: NON_VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: true } } }
I20260811 18:48:21.284879 24635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:21.286120 24635 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: attempting to promote NON_VOTER 915c1c7f244d4ebeaf057569e2162b01 to VOTER
I20260811 18:48:21.286264 24635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } }
I20260811 18:48:21.286698 23387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Refusing update from remote peer 5abd1ef3b2954060a1020004d81f4bf1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 18:48:21.286739 23740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 LEARNER]: Refusing update from remote peer 5abd1ef3b2954060a1020004d81f4bf1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 18:48:21.286976 24553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 18:48:21.287290 24553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 18:48:21.287801 24636 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.287943 23387 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.288100 23740 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.288858 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.288932 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:21.288887 24553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.288975 24553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:21.288931 23786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.289001 23786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:48:21.289004 24636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.289047 24636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:21.290112 24652 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5abd1ef3b2954060a1020004d81f4bf1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:21.290226 24652 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:48:21.293320 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.289787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:33427 (local address 127.0.0.1:57214)
0811 18:48:21.289957 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.289969 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.290022 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.290520 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.290520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.290522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:21.290575 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.290577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.291427 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.291428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.292278 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.292280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.292962 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.292963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.293023 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.293028 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.293165 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.293228 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":19,"thread_start_us":98,"threads_started":1}
I20260811 18:48:21.293432 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.290189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.9.215.2:33427)
0811 18:48:21.290376 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.290380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.290390 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:21.290422 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.290424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.290438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.290511 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:21.290632 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.291346 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.292350 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.292880 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.292947 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.293005 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.293191 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.293240 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":112,"threads_started":1}
I20260811 18:48:21.294301 24201 tablet_service.cc:1467] Processing CreateTablet for tablet 846410480e144a0485e28ad7912abee9 (DEFAULT_TABLE table=test-table [id=2463877d3207484b9e0c7b9d399a4a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:21.294725 24201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846410480e144a0485e28ad7912abee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:21.295550 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.290014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.0.0.1:42084)
0811 18:48:21.290126 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.290134 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.290172 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.290912 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.290913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.290915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:21.290952 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.290953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.293204 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.293205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.294287 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.294290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.295289 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.295291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.295348 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.295352 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.295397 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.295435 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":76,"threads_started":1}
I20260811 18:48:21.295709 24662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.290522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.9.215.1:43179)
0811 18:48:21.290747 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.290751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.290761 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:21.290797 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.290799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.290814 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.291022 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:21.291082 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.292175 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.294381 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.295180 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.295268 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.295445 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.295514 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.295559 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260811 18:48:21.296787 23925 tablet_service.cc:1467] Processing CreateTablet for tablet 846410480e144a0485e28ad7912abee9 (DEFAULT_TABLE table=test-table [id=2463877d3207484b9e0c7b9d399a4a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:21.297155 23925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846410480e144a0485e28ad7912abee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:21.300043 24663 tablet_bootstrap.cc:492] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Bootstrap starting.
I20260811 18:48:21.300128 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.289787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43959 (local address 127.0.0.1:58100)
0811 18:48:21.289930 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.289942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.289986 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.290557 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.290558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.290559 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:21.290606 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.290608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.292275 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.292276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.293031 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.293033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.299828 (+ 6795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.299830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.299924 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.299930 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.299966 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.300035 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 18:48:21.301081 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.289893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.9.215.3:43959)
0811 18:48:21.290373 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.290379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.290389 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:21.290430 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.290433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.290448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.290681 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:21.290746 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.292177 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.293119 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.293872 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.300568 (+ 6696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.300814 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.300862 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.300926 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":93,"threads_started":1}
I20260811 18:48:21.301451 24663 tablet_bootstrap.cc:654] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:21.302093 24664 tablet_bootstrap.cc:492] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Bootstrap starting.
I20260811 18:48:21.301966 24475 tablet_service.cc:1467] Processing CreateTablet for tablet 846410480e144a0485e28ad7912abee9 (DEFAULT_TABLE table=test-table [id=2463877d3207484b9e0c7b9d399a4a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:21.302368 24663 log.cc:824] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:21.302544 24475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 846410480e144a0485e28ad7912abee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:21.303318 24664 tablet_bootstrap.cc:654] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:21.303696 24663 tablet_bootstrap.cc:492] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: No bootstrap required, opened a new log
I20260811 18:48:21.303776 24663 ts_tablet_manager.cc:1397] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:48:21.304178 24664 log.cc:824] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:21.305122 24664 tablet_bootstrap.cc:492] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: No bootstrap required, opened a new log
I20260811 18:48:21.305192 24664 ts_tablet_manager.cc:1397] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:48:21.308714 24670 tablet_bootstrap.cc:492] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Bootstrap starting.
I20260811 18:48:21.308933 24664 raft_consensus.cc:348] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.309130 24664 raft_consensus.cc:374] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:21.309199 24664 raft_consensus.cc:729] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3948537b697d4d77a7946977d83a5ac1, State: Initialized, Role: FOLLOWER
I20260811 18:48:21.309360 24664 consensus_queue.cc:260] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.309785 24664 ts_tablet_manager.cc:1428] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 18:48:21.309733 24663 raft_consensus.cc:348] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.309916 24663 raft_consensus.cc:374] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:21.309959 24663 raft_consensus.cc:729] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581b5d0e6de143b4a31bf7e2834108aa, State: Initialized, Role: FOLLOWER
I20260811 18:48:21.310067 24066 heartbeater.cc:503] Master 127.9.215.62:37933 was elected leader, sending a full tablet report...
I20260811 18:48:21.310092 24663 consensus_queue.cc:260] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.310451 24663 ts_tablet_manager.cc:1428] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:48:21.310673 24670 tablet_bootstrap.cc:654] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:21.311517 24341 heartbeater.cc:503] Master 127.9.215.62:37933 was elected leader, sending a full tablet report...
I20260811 18:48:21.311882 24670 log.cc:824] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:21.312958 24670 tablet_bootstrap.cc:492] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: No bootstrap required, opened a new log
I20260811 18:48:21.313024 24670 ts_tablet_manager.cc:1397] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 18:48:21.317165 24670 raft_consensus.cc:348] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.317344 24670 raft_consensus.cc:374] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:21.317366 24670 raft_consensus.cc:729] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30511577f0b74d5094e9f9381c4fc97c, State: Initialized, Role: FOLLOWER
I20260811 18:48:21.317489 24670 consensus_queue.cc:260] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.317788 24670 ts_tablet_manager.cc:1428] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 18:48:21.317929 24601 heartbeater.cc:503] Master 127.9.215.62:37933 was elected leader, sending a full tablet report...
W20260811 18:48:21.334285 24343 tablet.cc:2369] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:48:21.393409 24068 tablet.cc:2369] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:48:21.506117 24604 tablet.cc:2369] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:48:21.552691 24677 raft_consensus.cc:482] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:21.552868 24677 raft_consensus.cc:504] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.553373 24677 leader_election.cc:288] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179), 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:21.556593 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.553578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.9.215.2:52461)
0811 18:48:21.553677 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.553691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.553760 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.553899 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.553900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.553901 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:21.554154 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.554156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.554820 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.554822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.555681 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.555683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.556270 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.556272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.556399 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.556403 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.556454 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.556504 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:48:21.556587 24662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.553683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:52461 (local address 127.9.215.1:43179)
0811 18:48:21.553737 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.553739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.553750 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:21.553831 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.553832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.553844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.553885 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:21.554245 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.554747 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.555766 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.556194 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.556260 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.556273 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.556473 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.556519 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:48:21.556794 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.553650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:56935 (local address 127.9.215.3:43959)
0811 18:48:21.553722 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.553724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.553797 (+ 73us) server_negotiation.cc:407] Connection header received
0811 18:48:21.553857 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.553858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.553871 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.553916 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:21.554051 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.554640 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.555741 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.556342 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.556407 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.556418 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.556683 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.556730 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:48:21.556849 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.553582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43959 (local address 127.9.215.2:56935)
0811 18:48:21.553738 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:21.553749 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:21.553802 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:21.553926 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:21.553927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:21.553928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:21.553994 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.553996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.554726 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.554727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.555640 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:21.555642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.556417 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.556418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:21.556607 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.556610 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:21.556670 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:48:21.556718 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":106}
I20260811 18:48:21.557273 24509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846410480e144a0485e28ad7912abee9" candidate_uuid: "581b5d0e6de143b4a31bf7e2834108aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30511577f0b74d5094e9f9381c4fc97c" is_pre_election: true
I20260811 18:48:21.557296 23967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846410480e144a0485e28ad7912abee9" candidate_uuid: "581b5d0e6de143b4a31bf7e2834108aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3948537b697d4d77a7946977d83a5ac1" is_pre_election: true
I20260811 18:48:21.557447 23967 raft_consensus.cc:2454] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 581b5d0e6de143b4a31bf7e2834108aa in term 0.
I20260811 18:48:21.557446 24509 raft_consensus.cc:2454] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 581b5d0e6de143b4a31bf7e2834108aa in term 0.
I20260811 18:48:21.557766 24109 leader_election.cc:302] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3948537b697d4d77a7946977d83a5ac1, 581b5d0e6de143b4a31bf7e2834108aa; no voters:
I20260811 18:48:21.557967 24677 raft_consensus.cc:2781] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:21.558027 24677 raft_consensus.cc:482] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:21.558050 24677 raft_consensus.cc:3037] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:21.559355 24677 raft_consensus.cc:504] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.559533 24677 leader_election.cc:288] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [CANDIDATE]: Term 1 election: Requested vote from peers 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179), 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:21.559720 23967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846410480e144a0485e28ad7912abee9" candidate_uuid: "581b5d0e6de143b4a31bf7e2834108aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3948537b697d4d77a7946977d83a5ac1"
I20260811 18:48:21.559755 24509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "846410480e144a0485e28ad7912abee9" candidate_uuid: "581b5d0e6de143b4a31bf7e2834108aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30511577f0b74d5094e9f9381c4fc97c"
I20260811 18:48:21.559808 23967 raft_consensus.cc:3037] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:21.559835 24509 raft_consensus.cc:3037] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:21.560899 23967 raft_consensus.cc:2454] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 581b5d0e6de143b4a31bf7e2834108aa in term 1.
I20260811 18:48:21.561115 24109 leader_election.cc:302] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3948537b697d4d77a7946977d83a5ac1, 581b5d0e6de143b4a31bf7e2834108aa; no voters:
I20260811 18:48:21.561190 24509 raft_consensus.cc:2454] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 581b5d0e6de143b4a31bf7e2834108aa in term 1.
I20260811 18:48:21.561204 24677 raft_consensus.cc:2781] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:21.561331 24677 raft_consensus.cc:686] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 LEADER]: Becoming Leader. State: Replica: 581b5d0e6de143b4a31bf7e2834108aa, State: Running, Role: LEADER
I20260811 18:48:21.561523 24677 consensus_queue.cc:237] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } }
I20260811 18:48:21.562491 22209 catalog_manager.cc:5495] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2). New cstate: current_term: 1 leader_uuid: "581b5d0e6de143b4a31bf7e2834108aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } health_report { overall_health: UNKNOWN } } }
I20260811 18:48:21.583623 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.581324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.9.215.2:33427)
0811 18:48:21.581385 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.581387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.581395 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:21.581436 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.581436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.581458 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.581521 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:21.581635 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.582107 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.582891 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.583093 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.583147 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.583292 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:21.583399 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:21.583486 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.583524 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.583545 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:48:21.586092 23967 raft_consensus.cc:1261] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Refusing update from remote peer 581b5d0e6de143b4a31bf7e2834108aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:21.586091 24509 raft_consensus.cc:1261] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Refusing update from remote peer 581b5d0e6de143b4a31bf7e2834108aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:21.586555 24688 consensus_queue.cc:1035] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:21.586710 24677 consensus_queue.cc:1035] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:21.590044 24693 mvcc.cc:205] Tried to move back new op lower bound from 7317397920093478912 to 7317397919996747776. Current Snapshot: MvccSnapshot[applied={T|T < 7317397920093478912}]
I20260811 18:48:21.592374 24696 mvcc.cc:205] Tried to move back new op lower bound from 7317397920093478912 to 7317397919996747776. Current Snapshot: MvccSnapshot[applied={T|T < 7317397920093478912 or (T in {7317397920093478912})}]
I20260811 18:48:21.594811 22207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:21.594937 22207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:21.596633 22207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5abd1ef3b2954060a1020004d81f4bf1: Sending DeleteTablet for 3 replicas of tablet 846410480e144a0485e28ad7912abee9
I20260811 18:48:21.597009 24201 tablet_service.cc:1514] Processing DeleteTablet for tablet 846410480e144a0485e28ad7912abee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:21 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:57214
I20260811 18:48:21.597062 23925 tablet_service.cc:1514] Processing DeleteTablet for tablet 846410480e144a0485e28ad7912abee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:21 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:42084
I20260811 18:48:21.597096 24475 tablet_service.cc:1514] Processing DeleteTablet for tablet 846410480e144a0485e28ad7912abee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:21 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:58100
I20260811 18:48:21.597627 24707 tablet_replica.cc:330] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: stopping tablet replica
I20260811 18:48:21.598438 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22150
I20260811 18:48:21.621027 24707 raft_consensus.cc:2229] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:21.621300 24707 raft_consensus.cc:2258] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:21.621583 24706 tablet_replica.cc:330] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: stopping tablet replica
I20260811 18:48:21.621730 24706 raft_consensus.cc:2229] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:21.621881 24706 raft_consensus.cc:2258] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:21.622391 24706 ts_tablet_manager.cc:1905] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:21.624612 24706 ts_tablet_manager.cc:1918] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:21.624693 24706 log.cc:1196] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/846410480e144a0485e28ad7912abee9
I20260811 18:48:21.624935 24706 ts_tablet_manager.cc:1939] T 846410480e144a0485e28ad7912abee9 P 581b5d0e6de143b4a31bf7e2834108aa: Deleting consensus metadata
I20260811 18:48:21.627993 24707 ts_tablet_manager.cc:1905] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:48:21.628036 24105 connection.cc:535] server connection from 127.0.0.1:57214 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:48:21.630002 24711 tablet_replica.cc:330] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: stopping tablet replica
I20260811 18:48:21.630151 24711 raft_consensus.cc:2229] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:21.630302 24711 raft_consensus.cc:2258] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:21.632763 24707 ts_tablet_manager.cc:1918] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:21.632965 24707 log.cc:1196] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/846410480e144a0485e28ad7912abee9
I20260811 18:48:21.633347 24707 ts_tablet_manager.cc:1939] T 846410480e144a0485e28ad7912abee9 P 3948537b697d4d77a7946977d83a5ac1: Deleting consensus metadata
W20260811 18:48:21.633808 23843 connection.cc:441] server connection from 127.0.0.1:42084 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42084 (request call id 1) could send its response
I20260811 18:48:21.635641 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.597996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.9.215.60:39169)
0811 18:48:21.598036 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.598037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.598101 (+ 64us) server_negotiation.cc:407] Connection header received
0811 18:48:21.598222 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.598223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.598238 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.598291 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.622422 (+ 24131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.623159 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.630140 (+ 6981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.630477 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.630559 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.632328 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.632330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.632330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.632340 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.634087 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.634488 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.634489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.634489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.634546 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.634750 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.634751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.634751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.634882 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.634961 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.635418 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.635542 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:21.635962 24711 ts_tablet_manager.cc:1905] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:21.637996 24711 ts_tablet_manager.cc:1918] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:21.638111 24711 log.cc:1196] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/846410480e144a0485e28ad7912abee9
I20260811 18:48:21.638396 24711 ts_tablet_manager.cc:1939] T 846410480e144a0485e28ad7912abee9 P 30511577f0b74d5094e9f9381c4fc97c: Deleting consensus metadata
W20260811 18:48:21.638903 24380 connection.cc:441] server connection from 127.0.0.1:58100 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58100 (request call id 1) could send its response
I20260811 18:48:21.640064 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.630668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.9.215.60:39169)
0811 18:48:21.630712 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.630714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.630987 (+ 273us) server_negotiation.cc:407] Connection header received
0811 18:48:21.631091 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.631092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.631103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.631135 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.632596 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.633323 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.634448 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.634717 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.634762 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.636928 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.636930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.636930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.636937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.638793 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.639233 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.639234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.639234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.639293 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.639510 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.639512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.639512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.639664 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.639769 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.639887 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.639967 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:21.641595 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:21.634154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.9.215.61:35303)
0811 18:48:21.634198 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:21.634200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:21.634378 (+ 178us) server_negotiation.cc:407] Connection header received
0811 18:48:21.634423 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:21.634424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:21.634439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:21.634483 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:21.635224 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.635938 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.637203 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:21.637506 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:21.637572 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:21.638886 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:21.638887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:21.638887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:21.638895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:21.640482 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.640911 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.640912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.640913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.640974 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:21.641159 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:21.641160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:21.641160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:21.641275 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:21.641344 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:21.641459 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 18:48:21.641531 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:22.592052 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:22.591733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:51661)
0811 18:48:22.591897 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:22.591989 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260811 18:48:22.592926 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:22.592304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:60163)
0811 18:48:22.592674 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:22.592875 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":58,"threads_started":1}
W20260811 18:48:22.592950 24601 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:37933 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
W20260811 18:48:22.593293 24066 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:37933 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:22.593317 24731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:22.592549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:51545)
0811 18:48:22.593201 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:22.593273 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":69,"threads_started":1}
W20260811 18:48:22.593506 24341 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:37933 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:23.097122 24732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5abd1ef3b2954060a1020004d81f4bf1)
I20260811 18:48:23.097235 24732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } }
I20260811 18:48:23.097419 24732 leader_election.cc:288] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933), cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61:35303)
I20260811 18:48:23.097813 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.097573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46412)
0811 18:48:23.097707 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.097768 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":46,"threads_started":1}
W20260811 18:48:23.097985 23692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:23.098390 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.098285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46428)
0811 18:48:23.098333 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.098363 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:48:23.098374 24744 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5abd1ef3b2954060a1020004d81f4bf1)
W20260811 18:48:23.098469 23692 leader_election.cc:334] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933): Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:23.098620 24744 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } }
I20260811 18:48:23.098855 24744 leader_election.cc:288] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933), 915c1c7f244d4ebeaf057569e2162b01 (127.9.215.60:39169)
I20260811 18:48:23.099116 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.098944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46436)
0811 18:48:23.099027 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.099078 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":43,"threads_started":1}
W20260811 18:48:23.099331 23329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:23.099807 24746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.099688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46438)
0811 18:48:23.099737 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.099779 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 18:48:23.099931 23329 leader_election.cc:334] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933): Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:23.101023 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.097633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:35303 (local address 127.0.0.1:57072)
0811 18:48:23.097750 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.097761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.097820 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.097967 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.097968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.097970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.098027 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.098029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.098870 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.098871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.100040 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.100043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.100785 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.100786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.100871 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.100876 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.100930 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.100972 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260811 18:48:23.101061 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.097713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.9.215.61:35303)
0811 18:48:23.097831 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.097833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.097844 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:23.097877 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.097878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.097889 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.097963 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.098092 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.098797 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.100118 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.100672 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.100746 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.100760 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.100942 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.100986 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260811 18:48:23.101260 23387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "915c1c7f244d4ebeaf057569e2162b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfb18b42f7b545f185f22a46bcf9267d" is_pre_election: true
I20260811 18:48:23.101365 23387 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915c1c7f244d4ebeaf057569e2162b01 in term 1.
I20260811 18:48:23.101500 23694 leader_election.cc:302] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 915c1c7f244d4ebeaf057569e2162b01, cfb18b42f7b545f185f22a46bcf9267d; no voters: 5abd1ef3b2954060a1020004d81f4bf1
I20260811 18:48:23.101593 24732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:48:23.101639 24732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5abd1ef3b2954060a1020004d81f4bf1)
I20260811 18:48:23.101663 24732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:48:23.102309 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.099070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:39169 (local address 127.0.0.1:39590)
0811 18:48:23.099198 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.099207 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.099259 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.099495 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.099497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.099500 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.099559 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.099562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.100350 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.100351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.101379 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.101381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.102063 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.102064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.102144 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.102149 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.102195 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.102242 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":51,"threads_started":1}
I20260811 18:48:23.102309 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.099226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.9.215.60:39169)
0811 18:48:23.099367 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.099368 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.099374 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:48:23.099393 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.099393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.099402 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.099493 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.099651 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.100289 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.101502 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.101991 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.102057 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.102070 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.102207 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.102249 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":91,"threads_started":1}
I20260811 18:48:23.102447 24732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } }
I20260811 18:48:23.102586 24732 leader_election.cc:288] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 2 election: Requested vote from peers 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933), cfb18b42f7b545f185f22a46bcf9267d (127.9.215.61:35303)
I20260811 18:48:23.102702 23740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfb18b42f7b545f185f22a46bcf9267d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "915c1c7f244d4ebeaf057569e2162b01" is_pre_election: true
I20260811 18:48:23.102766 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.102670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46454)
0811 18:48:23.102708 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.102744 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 18:48:23.102784 23387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "915c1c7f244d4ebeaf057569e2162b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfb18b42f7b545f185f22a46bcf9267d"
I20260811 18:48:23.102831 23740 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cfb18b42f7b545f185f22a46bcf9267d in current term 2: Already voted for candidate 915c1c7f244d4ebeaf057569e2162b01 in this term.
I20260811 18:48:23.102866 23387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:48:23.103022 23329 leader_election.cc:302] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfb18b42f7b545f185f22a46bcf9267d; no voters: 5abd1ef3b2954060a1020004d81f4bf1, 915c1c7f244d4ebeaf057569e2162b01
I20260811 18:48:23.103590 23387 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915c1c7f244d4ebeaf057569e2162b01 in term 2.
I20260811 18:48:23.103695 24744 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:48:23.103749 23694 leader_election.cc:302] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915c1c7f244d4ebeaf057569e2162b01, cfb18b42f7b545f185f22a46bcf9267d; no voters:
I20260811 18:48:23.103830 24732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:48:23.103952 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.103865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46464)
0811 18:48:23.103900 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.103932 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 18:48:23.103997 24732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [term 2 LEADER]: Becoming Leader. State: Replica: 915c1c7f244d4ebeaf057569e2162b01, State: Running, Role: LEADER
W20260811 18:48:23.104029 23692 leader_election.cc:334] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933): Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
I20260811 18:48:23.104074 24732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } }
I20260811 18:48:23.104408 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 915c1c7f244d4ebeaf057569e2162b01. Latest consensus state: current_term: 2 leader_uuid: "915c1c7f244d4ebeaf057569e2162b01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:23.104481 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:23.104642 24758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:23.105137 24758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:23.105224 24758 catalog_manager.cc:1259] Loaded cluster ID: b6358a463d964b94bea1b908646655ad
I20260811 18:48:23.105257 24758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:23.105415 24758 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:23.105489 24758 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Loaded TSK: 0
I20260811 18:48:23.105621 24758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:23.123579 23717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:23.125463 23717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:23.126670 23387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [term 2 FOLLOWER]: Refusing update from remote peer 915c1c7f244d4ebeaf057569e2162b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 18:48:23.126947 24750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 18:48:23.127674 24744 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 915c1c7f244d4ebeaf057569e2162b01. Latest consensus state: current_term: 2 leader_uuid: "915c1c7f244d4ebeaf057569e2162b01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:23.127784 24744 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:48:23.127981 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.127708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46482)
0811 18:48:23.127907 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.127946 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 18:48:23.128312 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.128204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.0.0.1:46496)
0811 18:48:23.128267 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.128290 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 18:48:23.128396 23692 consensus_peers.cc:487] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 -> Peer 5abd1ef3b2954060a1020004d81f4bf1 (127.9.215.62:37933): Couldn't send request to peer 5abd1ef3b2954060a1020004d81f4bf1. Status: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:48:23.129066 24732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "915c1c7f244d4ebeaf057569e2162b01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:23.129160 24732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:23.129241 24744 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "915c1c7f244d4ebeaf057569e2162b01" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "5abd1ef3b2954060a1020004d81f4bf1" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37933 } } peers { permanent_uuid: "cfb18b42f7b545f185f22a46bcf9267d" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 35303 } attrs { promote: false } } peers { permanent_uuid: "915c1c7f244d4ebeaf057569e2162b01" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 39169 } attrs { promote: false } } }
I20260811 18:48:23.129323 24744 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb18b42f7b545f185f22a46bcf9267d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:48:23.129479 24762 mvcc.cc:205] Tried to move back new op lower bound from 7317397926404497408 to 7317397926314844160. Current Snapshot: MvccSnapshot[applied={T|T < 7317397926404497408 or (T in {7317397926404497408})}]
I20260811 18:48:23.134501 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.131343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:33427 (local address 127.0.0.1:37952)
0811 18:48:23.131376 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.131388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.131433 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.131673 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.131674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.131676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.131729 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.131731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.132549 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.132549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.133534 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.133537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.134272 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.134329 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.134335 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.134382 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.134423 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 18:48:23.134509 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.131425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.9.215.2:33427)
0811 18:48:23.131562 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.131564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.131571 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:23.131598 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.131599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.131609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.131670 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.131814 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.132475 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.133623 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.134170 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134271 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.134285 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.134395 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.134434 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
I20260811 18:48:23.134923 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.131661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.9.215.1:43179)
0811 18:48:23.131787 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.131789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.131797 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:23.131821 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.131822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.131832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.131885 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.132023 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.132843 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.133998 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.134531 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134576 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.134588 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.134807 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.134849 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260811 18:48:23.135039 24201 tablet_service.cc:1467] Processing CreateTablet for tablet 40ed4fc475724045bad60340daa94db9 (DEFAULT_TABLE table=test-table [id=0cff89e63c9742f1be390f0b9e7f4080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:23.135061 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.131385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43959 (local address 127.0.0.1:59090)
0811 18:48:23.131494 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.131505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.131554 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.132124 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.132125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.132128 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.132162 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.132164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.133115 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.133116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.134165 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.134167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.134841 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.134902 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.134905 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.134949 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.134994 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 18:48:23.135154 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.131432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.0.0.1:57370)
0811 18:48:23.131684 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.131693 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.131734 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.131893 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.131894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.131896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.131943 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.131945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.132933 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.132934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.133896 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.133899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.134651 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.134712 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.134727 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.135003 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.135052 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":181,"threads_started":1}
I20260811 18:48:23.135182 24201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40ed4fc475724045bad60340daa94db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:23.135288 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.131833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.9.215.3:43959)
0811 18:48:23.131995 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.131997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.132004 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:23.132028 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.132029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.132040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.132218 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.132268 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.132979 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134235 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.134771 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.134828 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.134840 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.135168 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.135217 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":57,"threads_started":1}
I20260811 18:48:23.135382 23925 tablet_service.cc:1467] Processing CreateTablet for tablet 40ed4fc475724045bad60340daa94db9 (DEFAULT_TABLE table=test-table [id=0cff89e63c9742f1be390f0b9e7f4080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:23.135522 23925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40ed4fc475724045bad60340daa94db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:23.135521 24475 tablet_service.cc:1467] Processing CreateTablet for tablet 40ed4fc475724045bad60340daa94db9 (DEFAULT_TABLE table=test-table [id=0cff89e63c9742f1be390f0b9e7f4080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:23.135655 24475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40ed4fc475724045bad60340daa94db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:23.136799 24773 tablet_bootstrap.cc:492] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Bootstrap starting.
I20260811 18:48:23.137033 24775 tablet_bootstrap.cc:492] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Bootstrap starting.
I20260811 18:48:23.137182 24773 tablet_bootstrap.cc:654] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:23.137383 24775 tablet_bootstrap.cc:654] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:23.137494 24774 tablet_bootstrap.cc:492] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1: Bootstrap starting.
I20260811 18:48:23.137792 24773 tablet_bootstrap.cc:492] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: No bootstrap required, opened a new log
I20260811 18:48:23.137838 24773 ts_tablet_manager.cc:1397] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:23.137940 24773 raft_consensus.cc:348] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.138010 24773 raft_consensus.cc:374] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:23.138025 24773 raft_consensus.cc:729] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581b5d0e6de143b4a31bf7e2834108aa, State: Initialized, Role: FOLLOWER
I20260811 18:48:23.138063 24773 consensus_queue.cc:260] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.138105 24774 tablet_bootstrap.cc:654] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:23.138209 24773 ts_tablet_manager.cc:1428] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:23.138782 24774 tablet_bootstrap.cc:492] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1: No bootstrap required, opened a new log
I20260811 18:48:23.138657 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.138439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:44025)
0811 18:48:23.138578 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.138620 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":90,"threads_started":1}
I20260811 18:48:23.138841 24774 ts_tablet_manager.cc:1397] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:23.139037 24774 raft_consensus.cc:348] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.139102 24774 raft_consensus.cc:374] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:23.139118 24774 raft_consensus.cc:729] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3948537b697d4d77a7946977d83a5ac1, State: Initialized, Role: FOLLOWER
I20260811 18:48:23.139171 24774 consensus_queue.cc:260] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.139318 24774 ts_tablet_manager.cc:1428] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:23.139767 24339 heartbeater.cc:503] Master 127.9.215.60:39169 was elected leader, sending a full tablet report...
I20260811 18:48:23.139827 24775 tablet_bootstrap.cc:492] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: No bootstrap required, opened a new log
I20260811 18:48:23.139870 24775 ts_tablet_manager.cc:1397] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:48:23.139926 24065 heartbeater.cc:503] Master 127.9.215.60:39169 was elected leader, sending a full tablet report...
I20260811 18:48:23.140026 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.139546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:48895)
0811 18:48:23.139942 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.139983 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":89,"threads_started":1}
I20260811 18:48:23.140000 24775 raft_consensus.cc:348] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.140066 24775 raft_consensus.cc:374] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:23.140085 24775 raft_consensus.cc:729] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30511577f0b74d5094e9f9381c4fc97c, State: Initialized, Role: FOLLOWER
I20260811 18:48:23.140130 24775 consensus_queue.cc:260] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.140285 24775 ts_tablet_manager.cc:1428] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:23.140455 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.140237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:57943)
0811 18:48:23.140372 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.140411 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260811 18:48:23.140499 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.140133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:42045)
0811 18:48:23.140451 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.140477 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 18:48:23.141067 24600 heartbeater.cc:503] Master 127.9.215.60:39169 was elected leader, sending a full tablet report...
I20260811 18:48:23.141155 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.140534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:49685)
0811 18:48:23.140658 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.140695 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260811 18:48:23.141491 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.141405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:56547)
0811 18:48:23.141438 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.141464 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22,"threads_started":1}
I20260811 18:48:23.175549 24788 raft_consensus.cc:482] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:23.175704 24788 raft_consensus.cc:504] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.175733 24786 raft_consensus.cc:482] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:23.175827 24786 raft_consensus.cc:504] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.176072 24788 leader_election.cc:288] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427), 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179)
I20260811 18:48:23.176133 24786 leader_election.cc:288] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427), 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:23.179536 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:50827 (local address 127.9.215.2:33427)
0811 18:48:23.176543 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.176545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.176554 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:23.176584 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.176585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.176597 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.176653 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.176872 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177642 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.178771 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.179149 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.179209 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.179224 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.179378 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.179434 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:48:23.180115 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:33427 (local address 127.9.215.3:50827)
0811 18:48:23.176365 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.176375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.176427 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.176708 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.176709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.176711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.176790 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.176793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177761 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.177762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.178681 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.178684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.179224 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.179225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.179312 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.179315 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.179365 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.179449 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":32}
I20260811 18:48:23.180394 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:40853 (local address 127.9.215.3:43959)
0811 18:48:23.176707 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.176708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.176717 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:23.176737 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.176738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.176748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.176798 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.177049 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177806 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.179313 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.179933 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.180000 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.180011 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.180293 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.180334 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":289}
I20260811 18:48:23.180444 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:33427 (local address 127.9.215.1:36017)
0811 18:48:23.176374 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.176385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.176437 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.176872 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.176873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.176875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.176968 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.176970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177907 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.177908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.179233 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.179236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.180212 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.180213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.180293 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.180298 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.180330 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.180373 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":78}
I20260811 18:48:23.180408 24236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "30511577f0b74d5094e9f9381c4fc97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581b5d0e6de143b4a31bf7e2834108aa" is_pre_election: true
I20260811 18:48:23.180555 24236 raft_consensus.cc:2454] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30511577f0b74d5094e9f9381c4fc97c in term 0.
I20260811 18:48:23.180610 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43959 (local address 127.9.215.1:40853)
0811 18:48:23.176350 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.176364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.176426 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.176848 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.176850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.176852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.176968 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.176970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177903 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.177904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.179232 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.179236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.180016 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.180017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.180110 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.180116 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.180483 (+ 367us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.180536 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":138}
I20260811 18:48:23.180729 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:36017 (local address 127.9.215.2:33427)
0811 18:48:23.176748 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.176750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.176756 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:48:23.176777 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.176778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.176786 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.177035 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.177070 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177807 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.179521 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.180080 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.180598 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.180612 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.180629 (+ 17us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.180671 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":201,"thread_start_us":124,"threads_started":1}
I20260811 18:48:23.180758 24382 leader_election.cc:302] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30511577f0b74d5094e9f9381c4fc97c, 581b5d0e6de143b4a31bf7e2834108aa; no voters:
I20260811 18:48:23.180907 24236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "3948537b697d4d77a7946977d83a5ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581b5d0e6de143b4a31bf7e2834108aa" is_pre_election: true
I20260811 18:48:23.180953 24788 raft_consensus.cc:2781] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:23.180992 24788 raft_consensus.cc:482] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:23.180998 24236 raft_consensus.cc:2454] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3948537b697d4d77a7946977d83a5ac1 in term 0.
I20260811 18:48:23.181012 24788 raft_consensus.cc:3037] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:23.181073 24509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "3948537b697d4d77a7946977d83a5ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30511577f0b74d5094e9f9381c4fc97c" is_pre_election: true
I20260811 18:48:23.181282 23844 leader_election.cc:302] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3948537b697d4d77a7946977d83a5ac1, 581b5d0e6de143b4a31bf7e2834108aa; no voters:
I20260811 18:48:23.181424 24786 raft_consensus.cc:2781] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:23.181480 24786 raft_consensus.cc:482] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:23.181499 24786 raft_consensus.cc:3037] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:23.181943 24788 raft_consensus.cc:504] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.182098 24788 leader_election.cc:288] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [CANDIDATE]: Term 1 election: Requested vote from peers 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427), 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179)
I20260811 18:48:23.182099 24509 raft_consensus.cc:2379] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3948537b697d4d77a7946977d83a5ac1 in current term 1: Already voted for candidate 30511577f0b74d5094e9f9381c4fc97c in this term.
I20260811 18:48:23.182343 24786 raft_consensus.cc:504] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.182479 24786 leader_election.cc:288] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [CANDIDATE]: Term 1 election: Requested vote from peers 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427), 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959)
I20260811 18:48:23.182652 24509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "3948537b697d4d77a7946977d83a5ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30511577f0b74d5094e9f9381c4fc97c"
I20260811 18:48:23.182731 24509 raft_consensus.cc:2379] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3948537b697d4d77a7946977d83a5ac1 in current term 1: Already voted for candidate 30511577f0b74d5094e9f9381c4fc97c in this term.
I20260811 18:48:23.182811 24236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "3948537b697d4d77a7946977d83a5ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581b5d0e6de143b4a31bf7e2834108aa"
I20260811 18:48:23.182880 24236 raft_consensus.cc:3037] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:23.182874 24235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "30511577f0b74d5094e9f9381c4fc97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "581b5d0e6de143b4a31bf7e2834108aa"
I20260811 18:48:23.183585 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:55831 (local address 127.9.215.1:43179)
0811 18:48:23.176505 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.176506 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.176569 (+ 63us) server_negotiation.cc:407] Connection header received
0811 18:48:23.176634 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.176635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.176644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.176687 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:23.176954 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.177659 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.182763 (+ 5104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.183271 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.183323 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.183332 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.183505 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.183540 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:48:23.183629 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.176372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.9.215.3:55831)
0811 18:48:23.176519 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.176528 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:23.176578 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:23.176810 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:23.176811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:23.176813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:23.176861 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.176862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.179357 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.179358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.182663 (+ 3305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:23.182666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.183332 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.183333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:23.183404 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.183408 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:23.183498 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:48:23.183542 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 18:48:23.183792 24236 raft_consensus.cc:2454] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3948537b697d4d77a7946977d83a5ac1 in term 1.
I20260811 18:48:23.183867 23967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "30511577f0b74d5094e9f9381c4fc97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3948537b697d4d77a7946977d83a5ac1" is_pre_election: true
I20260811 18:48:23.183885 23965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40ed4fc475724045bad60340daa94db9" candidate_uuid: "30511577f0b74d5094e9f9381c4fc97c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3948537b697d4d77a7946977d83a5ac1"
I20260811 18:48:23.183946 23844 leader_election.cc:302] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3948537b697d4d77a7946977d83a5ac1, 581b5d0e6de143b4a31bf7e2834108aa; no voters: 30511577f0b74d5094e9f9381c4fc97c
I20260811 18:48:23.183964 23967 raft_consensus.cc:2379] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30511577f0b74d5094e9f9381c4fc97c in current term 1: Already voted for candidate 3948537b697d4d77a7946977d83a5ac1 in this term.
I20260811 18:48:23.184092 24786 raft_consensus.cc:2781] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:23.184099 24382 leader_election.cc:302] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30511577f0b74d5094e9f9381c4fc97c; no voters: 3948537b697d4d77a7946977d83a5ac1, 581b5d0e6de143b4a31bf7e2834108aa
I20260811 18:48:23.184183 24788 raft_consensus.cc:2726] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 18:48:23.184248 24786 raft_consensus.cc:686] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [term 1 LEADER]: Becoming Leader. State: Replica: 3948537b697d4d77a7946977d83a5ac1, State: Running, Role: LEADER
I20260811 18:48:23.184321 24786 consensus_queue.cc:237] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } }
I20260811 18:48:23.184582 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.184475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:51867)
0811 18:48:23.184523 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.184561 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 18:48:23.185045 23717 catalog_manager.cc:5495] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1). New cstate: current_term: 1 leader_uuid: "3948537b697d4d77a7946977d83a5ac1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3948537b697d4d77a7946977d83a5ac1" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } health_report { overall_health: HEALTHY } } }
I20260811 18:48:23.194521 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.192273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.9.215.1:43179)
0811 18:48:23.192349 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:23.192351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:23.192360 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:23.192396 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:23.192397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:23.192416 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:23.192483 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:23.192583 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.193059 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.193810 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:23.194007 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:23.194060 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:23.194168 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:23.194281 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:23.194335 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:23.194421 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 18:48:23.194442 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:48:23.195833 24236 raft_consensus.cc:1261] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Refusing update from remote peer 3948537b697d4d77a7946977d83a5ac1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:23.195865 24509 raft_consensus.cc:1261] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Refusing update from remote peer 3948537b697d4d77a7946977d83a5ac1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:23.196121 24794 consensus_queue.cc:1035] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30511577f0b74d5094e9f9381c4fc97c" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:23.196259 24786 consensus_queue.cc:1035] T 40ed4fc475724045bad60340daa94db9 P 3948537b697d4d77a7946977d83a5ac1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "581b5d0e6de143b4a31bf7e2834108aa" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:23.197432 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.197312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:41453)
0811 18:48:23.197353 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.197393 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 18:48:23.198370 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.197490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:56891)
0811 18:48:23.198296 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.198335 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":785}
I20260811 18:48:23.198868 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.198770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.3:50585)
0811 18:48:23.198811 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.198836 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 18:48:23.199182 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.198210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:39181)
0811 18:48:23.199113 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.199146 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":876}
I20260811 18:48:23.199260 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.199170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.2:48679)
0811 18:48:23.199212 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.199234 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 18:48:23.199641 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.199548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37933 (local address 127.9.215.1:50885)
0811 18:48:23.199585 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.199617 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:37933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 18:48:23.201435 23712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:23.201516 23712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:23.202821 23712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 915c1c7f244d4ebeaf057569e2162b01: Sending DeleteTablet for 3 replicas of tablet 40ed4fc475724045bad60340daa94db9
I20260811 18:48:23.203213 24201 tablet_service.cc:1514] Processing DeleteTablet for tablet 40ed4fc475724045bad60340daa94db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:23 UTC) from {username='oryx', principal='oryx/127.9.215.60@KRBTEST.COM'} at 127.0.0.1:37952
I20260811 18:48:23.203279 23925 tablet_service.cc:1514] Processing DeleteTablet for tablet 40ed4fc475724045bad60340daa94db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:23 UTC) from {username='oryx', principal='oryx/127.9.215.60@KRBTEST.COM'} at 127.0.0.1:57370
I20260811 18:48:23.203836 24475 tablet_service.cc:1514] Processing DeleteTablet for tablet 40ed4fc475724045bad60340daa94db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:23 UTC) from {username='oryx', principal='oryx/127.9.215.60@KRBTEST.COM'} at 127.0.0.1:59090
I20260811 18:48:23.204157 24828 tablet_replica.cc:330] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: stopping tablet replica
I20260811 18:48:23.204317 24828 raft_consensus.cc:2229] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:23.204478 24828 raft_consensus.cc:2258] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:23.204487 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 23654
I20260811 18:48:23.224865 24828 ts_tablet_manager.cc:1905] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:23.227377 24828 ts_tablet_manager.cc:1918] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:23.227622 24828 log.cc:1196] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/40ed4fc475724045bad60340daa94db9
I20260811 18:48:23.227986 24828 ts_tablet_manager.cc:1939] T 40ed4fc475724045bad60340daa94db9 P 30511577f0b74d5094e9f9381c4fc97c: Deleting consensus metadata
I20260811 18:48:23.228523 23691 catalog_manager.cc:4841] TS 30511577f0b74d5094e9f9381c4fc97c (127.9.215.3:43959): tablet 40ed4fc475724045bad60340daa94db9 (table test-table [id=0cff89e63c9742f1be390f0b9e7f4080]) successfully deleted
I20260811 18:48:23.231732 24831 tablet_replica.cc:330] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: stopping tablet replica
I20260811 18:48:23.231815 24831 raft_consensus.cc:2229] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:23.231880 24831 raft_consensus.cc:2258] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:23.232234 24831 ts_tablet_manager.cc:1905] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:23.233774 24831 ts_tablet_manager.cc:1918] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:23.233834 24831 log.cc:1196] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/40ed4fc475724045bad60340daa94db9
I20260811 18:48:23.234018 24831 ts_tablet_manager.cc:1939] T 40ed4fc475724045bad60340daa94db9 P 581b5d0e6de143b4a31bf7e2834108aa: Deleting consensus metadata
I20260811 18:48:23.234426 23694 catalog_manager.cc:4841] TS 581b5d0e6de143b4a31bf7e2834108aa (127.9.215.2:33427): tablet 40ed4fc475724045bad60340daa94db9 (table test-table [id=0cff89e63c9742f1be390f0b9e7f4080]) successfully deleted
I20260811 18:48:23.236259 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:23.236136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.0.0.1:57384)
0811 18:48:23.236196 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:23.236227 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 18:48:23.236420 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 24078
W20260811 18:48:23.249366 23694 catalog_manager.cc:4584] TS 3948537b697d4d77a7946977d83a5ac1 (127.9.215.1:43179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40ed4fc475724045bad60340daa94db9: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43179: connect: Connection refused (error 111)
I20260811 18:48:23.257715 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 24349
I20260811 18:48:23.274986 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 23093
I20260811 18:48:23.287859 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 23478
2026-08-11T18:48:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5305 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:23 dist-test-slave-pww0 krb5kdc[24840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:23 dist-test-slave-pww0 krb5kdc[24840](info): set up 2 sockets
Aug 11 18:48:23 dist-test-slave-pww0 krb5kdc[24840](info): commencing operation
krb5kdc: starting...
W20260811 18:48:24.519238 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:24 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab.
Entry for principal oryx/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab.
I20260811 18:48:24.646972 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37445
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab
--principal=oryx/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:24.731298 24867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:24.731561 24867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:24.731597 24867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:24.732848 24867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:24.732898 24867 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:48:24.732915 24867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:24.732931 24867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:24.732947 24867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:24.732965 24867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:24.734673 24867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37445
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.62.keytab
--principal=oryx/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:24.734957 24867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:24.735201 24867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:24.740163 24879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:24.740181 24888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:24.740379 24867 server_base.cc:1034] running on GCE node
W20260811 18:48:24.740229 24882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:24.740720 24867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:24.741019 24867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:24.742184 24867 hybrid_clock.cc:648] HybridClock initialized: now 1786474104742166 us; error 27 us; skew 500 ppm
Aug 11 18:48:24 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474104, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:24.749188 24867 init.cc:377] Logged in from keytab as oryx/127.9.215.62@KRBTEST.COM (short username oryx)
I20260811 18:48:24.749763 24867 webserver.cc:460] Webserver started at http://127.9.215.62:41175/ using document root <none> and password file <none>
I20260811 18:48:24.750012 24867 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:24.750067 24867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:24.750206 24867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:24.751358 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "d4ce08a6f5474b3e8c4959fd23c9f219"
format_stamp: "Formatted at 2026-08-11 18:48:24 on dist-test-slave-pww0"
server_key: "80fde3c1c988a83633e8d72f8af1fc80"
server_key_iv: "c0989e9452daac3a5bf27581faf44049"
server_key_version: "encryptionkey@0"
I20260811 18:48:24.751765 24867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "d4ce08a6f5474b3e8c4959fd23c9f219"
format_stamp: "Formatted at 2026-08-11 18:48:24 on dist-test-slave-pww0"
server_key: "80fde3c1c988a83633e8d72f8af1fc80"
server_key_iv: "c0989e9452daac3a5bf27581faf44049"
server_key_version: "encryptionkey@0"
I20260811 18:48:24.753185 24867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:24.754012 24900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:24.754246 24867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:24.754351 24867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "d4ce08a6f5474b3e8c4959fd23c9f219"
format_stamp: "Formatted at 2026-08-11 18:48:24 on dist-test-slave-pww0"
server_key: "80fde3c1c988a83633e8d72f8af1fc80"
server_key_iv: "c0989e9452daac3a5bf27581faf44049"
server_key_version: "encryptionkey@0"
I20260811 18:48:24.754423 24867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:24.773778 24867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:24.776140 24867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:24.776306 24867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:24.781489 24867 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:37445
I20260811 18:48:24.781558 25012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:37445 every 8 connection(s)
I20260811 18:48:24.781914 24867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:24.782617 25013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:24.783093 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 24867
I20260811 18:48:24.783195 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:24.783406 10076 external_mini_cluster.cc:1428] Setting key aad7c9ebe3a2821c19c2fd05a0dbd6aa
I20260811 18:48:24.785714 25013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Bootstrap starting.
I20260811 18:48:24.786368 25013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:24.786645 25013 log.cc:824] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:24.787375 25013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: No bootstrap required, opened a new log
I20260811 18:48:24.789564 25013 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37445 } }
I20260811 18:48:24.789716 25013 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:24.789752 25013 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4ce08a6f5474b3e8c4959fd23c9f219, State: Initialized, Role: FOLLOWER
I20260811 18:48:24.789886 25013 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37445 } }
I20260811 18:48:24.789971 25013 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:24.790004 25013 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:24.790062 25013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 18:48:24 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:24.790712 25013 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37445 } }
I20260811 18:48:24.790824 25013 leader_election.cc:302] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4ce08a6f5474b3e8c4959fd23c9f219; no voters:
I20260811 18:48:24.791019 25013 leader_election.cc:288] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:24.791070 25023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:24.791215 25023 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [term 1 LEADER]: Becoming Leader. State: Replica: d4ce08a6f5474b3e8c4959fd23c9f219, State: Running, Role: LEADER
I20260811 18:48:24.791325 25013 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:24.791324 25023 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37445 } }
I20260811 18:48:24.791941 25025 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37445 } } }
I20260811 18:48:24.793113 25025 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:24.791867 25027 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4ce08a6f5474b3e8c4959fd23c9f219. Latest consensus state: current_term: 1 leader_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4ce08a6f5474b3e8c4959fd23c9f219" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37445 } } }
W20260811 18:48:24.793234 25060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:48:24.793304 25027 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219 [sys.catalog]: This master's current role is: LEADER
W20260811 18:48:24.793315 25060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:48:24.793430 25062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:24.793615 25062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:24.794915 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:24.784119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.9.215.62:37445)
0811 18:48:24.784293 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:24.784303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:24.784331 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:48:24.784857 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:24.784866 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:24.784921 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:24.785050 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:24.785851 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.786869 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.787714 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.788318 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.788446 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:24.790750 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:24.790754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:24.790755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:24.790769 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:24.792981 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:24.793404 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:24.793408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:24.793408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:24.793509 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:24.793734 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:24.793735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:24.793735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:24.794037 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:24.794206 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:24.794572 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:48:24.794715 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260811 18:48:24.795444 25062 catalog_manager.cc:1347] Generated new cluster ID: d6b12b698d2a435eb6d35c4e3ba90704
I20260811 18:48:24.795686 25062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:24.819711 25062 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:24.820266 25062 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:24.826671 25062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Generated new TSK 0
I20260811 18:48:24.826896 25062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab.
Entry for principal oryx/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab.
I20260811 18:48:24.862794 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37445
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:24.949101 25069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:24.949350 25069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:24.949391 25069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:24.950711 25069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:24.950762 25069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:24.950791 25069 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:48:24.950817 25069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:24.952557 25069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:37445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:24.952828 25069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:24.953060 25069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:24.958455 25081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:24.958479 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:24.958455 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:24.958761 25069 server_base.cc:1034] running on GCE node
I20260811 18:48:24.959010 25069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:24.959321 25069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:24.960503 25069 hybrid_clock.cc:648] HybridClock initialized: now 1786474104960490 us; error 37 us; skew 500 ppm
Aug 11 18:48:24 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474104, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:24.963516 25069 init.cc:377] Logged in from keytab as oryx/127.9.215.1@KRBTEST.COM (short username oryx)
I20260811 18:48:24.964039 25069 webserver.cc:460] Webserver started at http://127.9.215.1:45533/ using document root <none> and password file <none>
I20260811 18:48:24.964263 25069 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:24.964318 25069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:24.964444 25069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:24.965533 25069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "fc71383dae124ee0bf1b6b0215d97071"
format_stamp: "Formatted at 2026-08-11 18:48:24 on dist-test-slave-pww0"
server_key: "ebf118a32e1cbe9f95900c21e1d0265c"
server_key_iv: "a3c000876df5938d6e92c4dbd4a6a9e3"
server_key_version: "encryptionkey@0"
I20260811 18:48:24.965895 25069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "fc71383dae124ee0bf1b6b0215d97071"
format_stamp: "Formatted at 2026-08-11 18:48:24 on dist-test-slave-pww0"
server_key: "ebf118a32e1cbe9f95900c21e1d0265c"
server_key_iv: "a3c000876df5938d6e92c4dbd4a6a9e3"
server_key_version: "encryptionkey@0"
I20260811 18:48:24.967315 25069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:24.968178 25094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:24.968338 25069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:24.968441 25069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "fc71383dae124ee0bf1b6b0215d97071"
format_stamp: "Formatted at 2026-08-11 18:48:24 on dist-test-slave-pww0"
server_key: "ebf118a32e1cbe9f95900c21e1d0265c"
server_key_iv: "a3c000876df5938d6e92c4dbd4a6a9e3"
server_key_version: "encryptionkey@0"
I20260811 18:48:24.968511 25069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:24.978224 25069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:24.980602 25069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:24.980758 25069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:24.981303 25069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:24.981348 25069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:24.981382 25069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:24.981407 25069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:24.988368 25069 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:35773
Aug 11 18:48:24 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474104, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:24.988443 25305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:35773 every 8 connection(s)
I20260811 18:48:24.988809 25069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:24.989403 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 25069
I20260811 18:48:24.989621 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:24.989820 10076 external_mini_cluster.cc:1428] Setting key c1db3289043694b5bfba260bcbfa0c76
I20260811 18:48:24.992233 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:24.981888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:56611 (local address 127.9.215.62:37445)
0811 18:48:24.981943 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:24.981945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:24.982151 (+ 206us) server_negotiation.cc:407] Connection header received
0811 18:48:24.982742 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:24.982744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:24.982757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:24.982823 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:24.983788 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.984580 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.985745 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.986089 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.986175 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:24.988977 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:24.988979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:24.988979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:24.988988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:24.990492 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:24.990973 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:24.990975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:24.990975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:24.991049 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:24.991378 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:24.991379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:24.991379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:24.991602 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:24.991663 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:24.992021 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 18:48:24.992114 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:48:24.992479 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:24.981855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.9.215.1:56611)
0811 18:48:24.982024 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:24.982065 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:24.982633 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:24.982943 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:24.982948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:24.983345 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:24.983689 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:24.983697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.984674 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.984675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:24.985652 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:24.985655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.986198 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.986199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:24.986352 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:24.986950 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:24.986962 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:24.988871 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:24.990562 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:24.990573 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:24.990575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:24.990888 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:24.991138 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:24.991138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:24.991139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:24.991318 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:24.991664 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:24.991672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:24.991882 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 18:48:24.992005 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:48:24.992224 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260811 18:48:24.993088 25308 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37445
I20260811 18:48:24.993193 25308 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:24.993443 25308 heartbeater.cc:511] Master 127.9.215.62:37445 requested a full tablet report, sending...
I20260811 18:48:24.994042 24928 ts_manager.cc:194] Registered new tserver with Master: fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773)
I20260811 18:48:24.994865 24928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:56611
WARNING: no policy specified for oryx/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:48:25.001873 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:24.994246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.9.215.1:36679)
0811 18:48:24.995578 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:24.995591 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:24.995682 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:24.995856 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:24.995857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:24.996082 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:24.996172 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:24.996175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.996839 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.996840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:24.997665 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:24.997668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.998014 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.998015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:24.998085 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:24.998562 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:24.998567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:24.998957 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:25.000623 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.000625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.000625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.000873 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.001135 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.001136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.001137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.001211 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.001552 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:25.001553 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:25.001632 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.001693 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.001791 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"thread_start_us":87,"threads_started":1}
I20260811 18:48:25.001878 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:24.994307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:36679 (local address 127.9.215.62:37445)
0811 18:48:24.994353 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:24.994355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:24.995651 (+ 1296us) server_negotiation.cc:407] Connection header received
0811 18:48:24.995750 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:24.995751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:24.995764 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:24.995793 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:24.996255 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.996738 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.997751 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:24.997942 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:24.998000 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:24.999041 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:24.999042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:24.999042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:24.999049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.000390 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.000967 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.000968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.000969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.001045 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.001319 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.001320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.001321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.001467 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.001542 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.001730 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.001811 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab.
Entry for principal oryx/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab.
I20260811 18:48:25.018646 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37445
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:25.102650 25326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:25.102883 25326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:25.102919 25326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:25.104133 25326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:25.104179 25326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:25.104208 25326 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:48:25.104233 25326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:25.105796 25326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:37445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:25.106050 25326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:25.106277 25326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:25.111434 25346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:25.111434 25343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:25.111444 25340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:25.111832 25326 server_base.cc:1034] running on GCE node
I20260811 18:48:25.112066 25326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:25.112334 25326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:25.113519 25326 hybrid_clock.cc:648] HybridClock initialized: now 1786474105113505 us; error 29 us; skew 500 ppm
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474105, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:25.116302 25326 init.cc:377] Logged in from keytab as oryx/127.9.215.2@KRBTEST.COM (short username oryx)
I20260811 18:48:25.116834 25326 webserver.cc:460] Webserver started at http://127.9.215.2:37427/ using document root <none> and password file <none>
I20260811 18:48:25.117049 25326 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:25.117096 25326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:25.117228 25326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:25.118304 25326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "7253e32dbdf34f02926a609942214d86"
format_stamp: "Formatted at 2026-08-11 18:48:25 on dist-test-slave-pww0"
server_key: "e862f2c21b10e2cd60f4bb8cac53644f"
server_key_iv: "93161d9ab4ecc76dd8e898027fd4788a"
server_key_version: "encryptionkey@0"
I20260811 18:48:25.118676 25326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "7253e32dbdf34f02926a609942214d86"
format_stamp: "Formatted at 2026-08-11 18:48:25 on dist-test-slave-pww0"
server_key: "e862f2c21b10e2cd60f4bb8cac53644f"
server_key_iv: "93161d9ab4ecc76dd8e898027fd4788a"
server_key_version: "encryptionkey@0"
I20260811 18:48:25.120083 25326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:25.120827 25365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.120970 25326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:25.121060 25326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "7253e32dbdf34f02926a609942214d86"
format_stamp: "Formatted at 2026-08-11 18:48:25 on dist-test-slave-pww0"
server_key: "e862f2c21b10e2cd60f4bb8cac53644f"
server_key_iv: "93161d9ab4ecc76dd8e898027fd4788a"
server_key_version: "encryptionkey@0"
I20260811 18:48:25.121130 25326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:25.141475 25326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:25.143790 25326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:25.143952 25326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:25.144603 25326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:25.144659 25326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.144690 25326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:25.144713 25326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474105, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:25.151804 25326 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:41439
I20260811 18:48:25.152176 25326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:25.152228 25609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:41439 every 8 connection(s)
I20260811 18:48:25.154245 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.145059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:60567 (local address 127.9.215.62:37445)
0811 18:48:25.145119 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.145121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.145401 (+ 280us) server_negotiation.cc:407] Connection header received
0811 18:48:25.145907 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.145909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.145922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.145973 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.146670 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.147310 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.148420 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.148654 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.148716 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.151314 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:25.151317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:25.151318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:25.151326 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.152904 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.153304 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.153306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.153306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.153372 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.153609 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.153611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.153611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.153756 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.153830 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.154077 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.154174 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:48:25.154371 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.145057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.9.215.2:60567)
0811 18:48:25.145230 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.145315 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.145809 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.146048 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.146051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.146364 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:25.146575 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.146581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.147405 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.147407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.148329 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.148333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.148746 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.148748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.148896 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.149497 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:25.149509 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:25.151204 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:25.152977 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.152980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.152982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.153207 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.153433 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.153434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.153435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.153528 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.153820 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:25.153824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:25.153952 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.154034 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.154179 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260811 18:48:25.154660 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 25326
I20260811 18:48:25.154765 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:25.154985 10076 external_mini_cluster.cc:1428] Setting key c248d8e8313ac8e74ade91a686794e65
I20260811 18:48:25.156090 25611 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37445
I20260811 18:48:25.156173 25611 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:25.156366 25611 heartbeater.cc:511] Master 127.9.215.62:37445 requested a full tablet report, sending...
I20260811 18:48:25.156764 24927 ts_manager.cc:194] Registered new tserver with Master: 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:25.157146 24927 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:60567
I20260811 18:48:25.163195 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.157781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:54657 (local address 127.9.215.62:37445)
0811 18:48:25.157829 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.157831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.157839 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:25.157862 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.157863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.157873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.157925 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.158288 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.158819 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.159533 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.159740 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.159816 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.160769 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:25.160770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:25.160771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:25.160779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.162064 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.162398 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.162400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.162400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.162454 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.162618 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.162619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.162619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.162783 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.162845 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.163003 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.163092 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:48:25.163208 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.156221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.9.215.2:54657)
0811 18:48:25.156378 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.156391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.156511 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.157947 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.157949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.158149 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:25.158213 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.158215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.158914 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.158915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.159426 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.159428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.159818 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.159819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.159894 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.160351 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:25.160355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:25.160680 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:25.162148 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.162149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.162149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.162331 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.162509 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.162510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.162510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.162569 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.162845 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:25.162847 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:25.162899 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.162971 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.163094 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab.
Entry for principal oryx/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab.
I20260811 18:48:25.182459 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:37445
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:25.267509 25624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:25.267742 25624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:25.267771 25624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:25.269009 25624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:25.269060 25624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:25.269081 25624 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:48:25.269099 25624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:25.270781 25624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.215.62:37445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:25.271037 25624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:25.271281 25624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:25.276638 25642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:25.276641 25648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:25.276638 25645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:25.276896 25624 server_base.cc:1034] running on GCE node
I20260811 18:48:25.277104 25624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:25.277334 25624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:25.278482 25624 hybrid_clock.cc:648] HybridClock initialized: now 1786474105278470 us; error 31 us; skew 500 ppm
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474105, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:25.281004 25624 init.cc:377] Logged in from keytab as oryx/127.9.215.3@KRBTEST.COM (short username oryx)
I20260811 18:48:25.281447 25624 webserver.cc:460] Webserver started at http://127.9.215.3:45939/ using document root <none> and password file <none>
I20260811 18:48:25.281654 25624 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:25.281704 25624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:25.281849 25624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:25.282941 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "32a2ef433ba84969a5e9ed940f134576"
format_stamp: "Formatted at 2026-08-11 18:48:25 on dist-test-slave-pww0"
server_key: "61f38d2462f4d93ce16dd2d1a884b423"
server_key_iv: "e05830a4c148cf237598f1fe8b8e6aa2"
server_key_version: "encryptionkey@0"
I20260811 18:48:25.283290 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "32a2ef433ba84969a5e9ed940f134576"
format_stamp: "Formatted at 2026-08-11 18:48:25 on dist-test-slave-pww0"
server_key: "61f38d2462f4d93ce16dd2d1a884b423"
server_key_iv: "e05830a4c148cf237598f1fe8b8e6aa2"
server_key_version: "encryptionkey@0"
I20260811 18:48:25.284808 25624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:25.285594 25662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.285766 25624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:25.285856 25624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "32a2ef433ba84969a5e9ed940f134576"
format_stamp: "Formatted at 2026-08-11 18:48:25 on dist-test-slave-pww0"
server_key: "61f38d2462f4d93ce16dd2d1a884b423"
server_key_iv: "e05830a4c148cf237598f1fe8b8e6aa2"
server_key_version: "encryptionkey@0"
I20260811 18:48:25.285924 25624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:25.298303 25624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:25.300303 25624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:25.300441 25624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:25.300951 25624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:25.300993 25624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.301028 25624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:25.301052 25624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474105, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:25.307621 25624 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:41189
I20260811 18:48:25.307760 25890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:41189 every 8 connection(s)
I20260811 18:48:25.308041 25624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:25.308492 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 25624
I20260811 18:48:25.308593 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:25.308804 10076 external_mini_cluster.cc:1428] Setting key 4bd9a70e48def316cb47f8fb82ae9e09
I20260811 18:48:25.310832 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.301539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:41497 (local address 127.9.215.62:37445)
0811 18:48:25.301592 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.301594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.301809 (+ 215us) server_negotiation.cc:407] Connection header received
0811 18:48:25.302384 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.302386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.302400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.302449 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.303311 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.304104 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.305140 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.305336 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.305394 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.308170 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:25.308172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:25.308172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:25.308181 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.309750 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.310114 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.310114 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.310115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.310167 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.310350 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.310351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.310351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.310456 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.310503 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.310706 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.310767 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:48:25.311036 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.301505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.9.215.3:41497)
0811 18:48:25.301686 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.301724 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.302278 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.302536 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.302540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.302903 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:25.303195 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.303202 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.304197 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.304198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.305050 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.305053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.305422 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.305423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.305555 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.306505 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:25.306514 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:25.308061 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:25.309827 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.309833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.309835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.310058 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.310207 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.310208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.310208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.310309 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.310494 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:25.310496 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:25.310612 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.310688 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.310816 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260811 18:48:25.311513 25891 heartbeater.cc:347] Connected to a master server at 127.9.215.62:37445
I20260811 18:48:25.311602 25891 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:25.311827 25891 heartbeater.cc:511] Master 127.9.215.62:37445 requested a full tablet report, sending...
I20260811 18:48:25.312217 24928 ts_manager.cc:194] Registered new tserver with Master: 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189)
I20260811 18:48:25.312683 24928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:41497
I20260811 18:48:25.317655 25900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.312489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.9.215.3:42213)
0811 18:48:25.312615 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.312625 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.312688 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.312840 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.312841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.313016 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:25.313086 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.313088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.313790 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.313791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.314333 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.314336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.314689 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.314690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.314734 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.315186 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:25.315191 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:25.315557 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:25.316845 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.316846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.316847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.317010 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.317166 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.317167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.317167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.317218 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.317414 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:25.317414 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:25.317454 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.317499 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.317584 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260811 18:48:25.317662 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.312569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:42213 (local address 127.9.215.62:37445)
0811 18:48:25.312609 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.312611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.312669 (+ 58us) server_negotiation.cc:407] Connection header received
0811 18:48:25.312741 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.312741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.312749 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.312800 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.313199 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.313681 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.314413 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.314631 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.314686 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.315632 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:25.315633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:25.315633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:25.315662 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.316773 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.317073 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.317074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.317074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.317122 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.317275 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.317275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.317276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.317373 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.317418 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.317516 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.317573 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:25.320595 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474105, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:48:25.342818 25021 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:25.337511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.9.215.62:37445)
0811 18:48:25.337578 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.337579 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.337597 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:48:25.337624 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.337625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.337634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.337679 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.338261 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.338763 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.339524 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.339718 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.339770 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.342728 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39834: BlockingRecv error: recv got EOF from 127.0.0.1:39834 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474105, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:25.350419 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.343595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.9.215.62:37445)
0811 18:48:25.343640 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.343642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.343728 (+ 86us) server_negotiation.cc:407] Connection header received
0811 18:48:25.343799 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.343800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.343810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.343857 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.344323 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.344814 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.345545 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.345734 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.345790 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.347968 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:25.347970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:25.347970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:25.347979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.349330 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.349673 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.349675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.349675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.349737 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.349902 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.349903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.349903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.350026 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.350118 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.350229 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.350322 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:25.352206 24928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:25.353852 24928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:25.361095 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.356546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.9.215.2:41439)
0811 18:48:25.356927 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.356931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.357045 (+ 114us) server_negotiation.cc:407] Connection header received
0811 18:48:25.357121 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.357124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.357142 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.357231 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.357532 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.358639 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.359887 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.360635 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.360714 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.360799 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.360944 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.360986 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":91,"threads_started":1}
I20260811 18:48:25.361101 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.356412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:41439 (local address 127.0.0.1:43684)
0811 18:48:25.356985 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.356994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.357044 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.357331 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.357332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.357335 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.357435 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.357437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.358727 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.358728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.359827 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.359829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.360724 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.360725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.360830 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.360842 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.360921 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.360999 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":50,"thread_start_us":99,"threads_started":1}
I20260811 18:48:25.361101 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.356629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:35773 (local address 127.0.0.1:36510)
0811 18:48:25.357002 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.357011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.357061 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.357361 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.357362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.357363 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.357431 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.357433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.358394 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.358395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.359744 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.359748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.360726 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.360727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.360824 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.360842 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.360921 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.360999 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":6,"thread_start_us":67,"threads_started":1}
I20260811 18:48:25.361410 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.356769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.9.215.1:35773)
0811 18:48:25.356925 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.356930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.357064 (+ 134us) server_negotiation.cc:407] Connection header received
0811 18:48:25.357210 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.357212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.357227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.357291 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.357511 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.358309 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.359834 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.360638 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.360714 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.360807 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.361248 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.361307 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260811 18:48:25.362295 25474 tablet_service.cc:1467] Processing CreateTablet for tablet 619ea29970f646c6933d5be3ece828b4 (DEFAULT_TABLE table=test-table [id=91289a6cc529414db390a2c82cbf420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:25.362593 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.356398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:41189 (local address 127.0.0.1:43824)
0811 18:48:25.356752 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.356800 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.356872 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.357322 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.357326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.357328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.357416 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.357418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.358760 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.358762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.359751 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.359754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.362359 (+ 2605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.362360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.362425 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.362430 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.362480 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.362526 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260811 18:48:25.362670 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619ea29970f646c6933d5be3ece828b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.362850 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.356623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.9.215.3:41189)
0811 18:48:25.356828 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.356833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.356847 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:48:25.357152 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.357155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.357170 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.357238 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.357476 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.358682 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.360845 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.362275 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.362342 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.362422 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.362714 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.362758 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
I20260811 18:48:25.363407 25762 tablet_service.cc:1467] Processing CreateTablet for tablet 619ea29970f646c6933d5be3ece828b4 (DEFAULT_TABLE table=test-table [id=91289a6cc529414db390a2c82cbf420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:25.363809 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619ea29970f646c6933d5be3ece828b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.363934 25173 tablet_service.cc:1467] Processing CreateTablet for tablet 619ea29970f646c6933d5be3ece828b4 (DEFAULT_TABLE table=test-table [id=91289a6cc529414db390a2c82cbf420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:25.364272 25173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619ea29970f646c6933d5be3ece828b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.366081 25941 tablet_bootstrap.cc:492] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Bootstrap starting.
I20260811 18:48:25.366698 25941 tablet_bootstrap.cc:654] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.366983 25941 log.cc:824] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:25.367238 25942 tablet_bootstrap.cc:492] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Bootstrap starting.
I20260811 18:48:25.368120 25948 tablet_bootstrap.cc:492] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Bootstrap starting.
I20260811 18:48:25.368153 25941 tablet_bootstrap.cc:492] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: No bootstrap required, opened a new log
I20260811 18:48:25.368225 25941 ts_tablet_manager.cc:1397] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:25.368237 25942 tablet_bootstrap.cc:654] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.368553 25942 log.cc:824] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:25.368860 25948 tablet_bootstrap.cc:654] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.369174 25948 log.cc:824] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:25.369374 25942 tablet_bootstrap.cc:492] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: No bootstrap required, opened a new log
I20260811 18:48:25.369444 25942 ts_tablet_manager.cc:1397] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:25.370025 25948 tablet_bootstrap.cc:492] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: No bootstrap required, opened a new log
I20260811 18:48:25.370106 25948 ts_tablet_manager.cc:1397] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:25.370004 25941 raft_consensus.cc:348] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.370173 25941 raft_consensus.cc:374] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.370215 25941 raft_consensus.cc:729] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.370347 25941 consensus_queue.cc:260] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.370636 25941 ts_tablet_manager.cc:1428] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:25.370661 25611 heartbeater.cc:503] Master 127.9.215.62:37445 was elected leader, sending a full tablet report...
I20260811 18:48:25.371501 25942 raft_consensus.cc:348] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.371688 25942 raft_consensus.cc:374] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.371728 25942 raft_consensus.cc:729] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.371870 25942 consensus_queue.cc:260] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.372154 25942 ts_tablet_manager.cc:1428] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:48:25.372301 25891 heartbeater.cc:503] Master 127.9.215.62:37445 was elected leader, sending a full tablet report...
I20260811 18:48:25.372725 25948 raft_consensus.cc:348] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.372877 25948 raft_consensus.cc:374] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.372915 25948 raft_consensus.cc:729] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71383dae124ee0bf1b6b0215d97071, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.373003 25948 consensus_queue.cc:260] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.373279 25948 ts_tablet_manager.cc:1428] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:48:25.373327 25308 heartbeater.cc:503] Master 127.9.215.62:37445 was elected leader, sending a full tablet report...
W20260811 18:48:25.402782 25612 tablet.cc:2369] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:48:25.489696 25309 tablet.cc:2369] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:48:25.559340 25893 tablet.cc:2369] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:48:25.612874 25956 raft_consensus.cc:482] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:25.613014 25956 raft_consensus.cc:504] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.613360 25956 leader_election.cc:288] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189), 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:25.616329 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.613731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:42961 (local address 127.9.215.2:41439)
0811 18:48:25.613776 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.613777 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.613788 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:25.613871 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.613872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.613884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.613935 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.614103 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.614632 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.615611 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.615984 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.616039 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.616048 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.616208 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.616252 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:48:25.616353 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.613690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:46933 (local address 127.9.215.3:41189)
0811 18:48:25.613735 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.613737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.613795 (+ 58us) server_negotiation.cc:407] Connection header received
0811 18:48:25.613868 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.613869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.613882 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.613931 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.614094 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.614581 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.615674 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.616020 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.616090 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.616100 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.616254 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.616293 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:48:25.616364 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.613578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:41439 (local address 127.9.215.1:42961)
0811 18:48:25.613717 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.613729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.613801 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.613951 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.613952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.613954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.614033 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.614034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.614720 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.614721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.615530 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.615533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.616067 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.616068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.616138 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.616142 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.616185 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.616224 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260811 18:48:25.616370 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.613583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:41189 (local address 127.9.215.1:46933)
0811 18:48:25.613725 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.613736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.613806 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.613945 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.613946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.613948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.614019 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.614021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.614660 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.614661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.615569 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.615572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.616105 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.616105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.616174 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.616179 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.616233 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.616273 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260811 18:48:25.616751 25516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619ea29970f646c6933d5be3ece828b4" candidate_uuid: "fc71383dae124ee0bf1b6b0215d97071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7253e32dbdf34f02926a609942214d86" is_pre_election: true
I20260811 18:48:25.616923 25516 raft_consensus.cc:2454] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc71383dae124ee0bf1b6b0215d97071 in term 0.
I20260811 18:48:25.616917 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619ea29970f646c6933d5be3ece828b4" candidate_uuid: "fc71383dae124ee0bf1b6b0215d97071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a2ef433ba84969a5e9ed940f134576" is_pre_election: true
I20260811 18:48:25.617079 25810 raft_consensus.cc:2454] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc71383dae124ee0bf1b6b0215d97071 in term 0.
I20260811 18:48:25.617177 25095 leader_election.cc:302] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7253e32dbdf34f02926a609942214d86, fc71383dae124ee0bf1b6b0215d97071; no voters:
I20260811 18:48:25.617321 25956 raft_consensus.cc:2781] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:25.617378 25956 raft_consensus.cc:482] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:25.617394 25956 raft_consensus.cc:3037] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.618124 25956 raft_consensus.cc:504] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.618263 25956 leader_election.cc:288] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [CANDIDATE]: Term 1 election: Requested vote from peers 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189), 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:25.618482 25516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619ea29970f646c6933d5be3ece828b4" candidate_uuid: "fc71383dae124ee0bf1b6b0215d97071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7253e32dbdf34f02926a609942214d86"
I20260811 18:48:25.618492 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619ea29970f646c6933d5be3ece828b4" candidate_uuid: "fc71383dae124ee0bf1b6b0215d97071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a2ef433ba84969a5e9ed940f134576"
I20260811 18:48:25.618585 25516 raft_consensus.cc:3037] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.618585 25810 raft_consensus.cc:3037] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.619570 25516 raft_consensus.cc:2454] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc71383dae124ee0bf1b6b0215d97071 in term 1.
I20260811 18:48:25.619570 25810 raft_consensus.cc:2454] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc71383dae124ee0bf1b6b0215d97071 in term 1.
I20260811 18:48:25.619793 25101 leader_election.cc:302] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, fc71383dae124ee0bf1b6b0215d97071; no voters:
I20260811 18:48:25.619906 25956 raft_consensus.cc:2781] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:25.620025 25956 raft_consensus.cc:686] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 1 LEADER]: Becoming Leader. State: Replica: fc71383dae124ee0bf1b6b0215d97071, State: Running, Role: LEADER
I20260811 18:48:25.620121 25956 consensus_queue.cc:237] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } }
I20260811 18:48:25.620955 24927 catalog_manager.cc:5495] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1). New cstate: current_term: 1 leader_uuid: "fc71383dae124ee0bf1b6b0215d97071" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } health_report { overall_health: HEALTHY } } }
I20260811 18:48:25.648335 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.645905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.9.215.1:35773)
0811 18:48:25.645973 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.645975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.645985 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:25.646102 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.646104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.646128 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.646196 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:25.646336 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.646823 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.647619 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.647810 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.647865 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.648015 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:25.648137 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:25.648211 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.648229 (+ 18us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.648254 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:48:25.650362 25810 raft_consensus.cc:1261] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Refusing update from remote peer fc71383dae124ee0bf1b6b0215d97071: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:25.650377 25516 raft_consensus.cc:1261] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Refusing update from remote peer fc71383dae124ee0bf1b6b0215d97071: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:25.650733 25961 consensus_queue.cc:1035] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:25.650892 25956 consensus_queue.cc:1035] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:25.653913 25972 mvcc.cc:205] Tried to move back new op lower bound from 7317397936740917248 to 7317397936620703744. Current Snapshot: MvccSnapshot[applied={T|T < 7317397936740917248}]
I20260811 18:48:25.656157 24928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:25.656299 24928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:25.657136 24928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Sending DeleteTablet for 3 replicas of tablet 619ea29970f646c6933d5be3ece828b4
I20260811 18:48:25.657555 25474 tablet_service.cc:1514] Processing DeleteTablet for tablet 619ea29970f646c6933d5be3ece828b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:25 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43684
I20260811 18:48:25.657569 25762 tablet_service.cc:1514] Processing DeleteTablet for tablet 619ea29970f646c6933d5be3ece828b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:25 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43824
I20260811 18:48:25.657614 25173 tablet_service.cc:1514] Processing DeleteTablet for tablet 619ea29970f646c6933d5be3ece828b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:25 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36510
I20260811 18:48:25.658083 25996 tablet_replica.cc:330] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: stopping tablet replica
I20260811 18:48:25.658036 24928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:25.658190 24928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:25.658208 25996 raft_consensus.cc:2229] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:25.658327 25996 raft_consensus.cc:2258] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:25.657958 25993 tablet_replica.cc:330] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: stopping tablet replica
I20260811 18:48:25.658547 25993 raft_consensus.cc:2229] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:25.658681 25993 raft_consensus.cc:2258] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:25.659775 25762 tablet_service.cc:1467] Processing CreateTablet for tablet bf06cead018b4595a0b7c8304f6c41e2 (DEFAULT_TABLE table=test-table [id=9601f6104cc44cadae39dbd1fc3f8123]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:25.659763 25173 tablet_service.cc:1467] Processing CreateTablet for tablet bf06cead018b4595a0b7c8304f6c41e2 (DEFAULT_TABLE table=test-table [id=9601f6104cc44cadae39dbd1fc3f8123]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:25.659762 25474 tablet_service.cc:1467] Processing CreateTablet for tablet bf06cead018b4595a0b7c8304f6c41e2 (DEFAULT_TABLE table=test-table [id=9601f6104cc44cadae39dbd1fc3f8123]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:25.659910 25173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf06cead018b4595a0b7c8304f6c41e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.659910 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf06cead018b4595a0b7c8304f6c41e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.659910 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf06cead018b4595a0b7c8304f6c41e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.661386 25948 tablet_bootstrap.cc:492] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Bootstrap starting.
I20260811 18:48:25.661384 25941 tablet_bootstrap.cc:492] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Bootstrap starting.
I20260811 18:48:25.661828 25941 tablet_bootstrap.cc:654] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.661830 25948 tablet_bootstrap.cc:654] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.662164 25942 tablet_bootstrap.cc:492] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Bootstrap starting.
I20260811 18:48:25.662449 25948 tablet_bootstrap.cc:492] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: No bootstrap required, opened a new log
I20260811 18:48:25.662489 25948 ts_tablet_manager.cc:1397] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:25.662614 25942 tablet_bootstrap.cc:654] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.662638 25948 raft_consensus.cc:348] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.662712 25948 raft_consensus.cc:374] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.662732 25948 raft_consensus.cc:729] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71383dae124ee0bf1b6b0215d97071, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.662782 25948 consensus_queue.cc:260] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.662864 25948 ts_tablet_manager.cc:1428] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 18:48:25.662981 25993 ts_tablet_manager.cc:1905] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:25.663527 25996 ts_tablet_manager.cc:1905] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:25.665485 25996 ts_tablet_manager.cc:1918] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:25.665586 25996 log.cc:1196] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/619ea29970f646c6933d5be3ece828b4
I20260811 18:48:25.666083 25996 ts_tablet_manager.cc:1939] T 619ea29970f646c6933d5be3ece828b4 P fc71383dae124ee0bf1b6b0215d97071: Deleting consensus metadata
I20260811 18:48:25.666654 24901 catalog_manager.cc:4841] TS fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773): tablet 619ea29970f646c6933d5be3ece828b4 (table test-table [id=91289a6cc529414db390a2c82cbf420d]) successfully deleted
I20260811 18:48:25.667685 25997 tablet_replica.cc:330] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: stopping tablet replica
I20260811 18:48:25.667848 25997 raft_consensus.cc:2229] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:25.667927 25941 tablet_bootstrap.cc:492] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: No bootstrap required, opened a new log
I20260811 18:48:25.668051 25941 ts_tablet_manager.cc:1397] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260811 18:48:25.668172 25997 raft_consensus.cc:2258] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:25.668279 25941 raft_consensus.cc:348] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.668362 25941 raft_consensus.cc:374] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.668381 25941 raft_consensus.cc:729] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.668449 25941 consensus_queue.cc:260] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.668546 25941 ts_tablet_manager.cc:1428] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.668680 25993 ts_tablet_manager.cc:1918] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:25.668776 25993 log.cc:1196] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/619ea29970f646c6933d5be3ece828b4
I20260811 18:48:25.668851 25997 ts_tablet_manager.cc:1905] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:25.669034 25993 ts_tablet_manager.cc:1939] T 619ea29970f646c6933d5be3ece828b4 P 7253e32dbdf34f02926a609942214d86: Deleting consensus metadata
I20260811 18:48:25.669497 25942 tablet_bootstrap.cc:492] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: No bootstrap required, opened a new log
I20260811 18:48:25.669553 25942 ts_tablet_manager.cc:1397] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260811 18:48:25.669708 25942 raft_consensus.cc:348] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.669775 25942 raft_consensus.cc:374] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.669754 24901 catalog_manager.cc:4841] TS 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439): tablet 619ea29970f646c6933d5be3ece828b4 (table test-table [id=91289a6cc529414db390a2c82cbf420d]) successfully deleted
I20260811 18:48:25.669795 25942 raft_consensus.cc:729] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.669842 25942 consensus_queue.cc:260] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.669929 25942 ts_tablet_manager.cc:1428] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.670544 25997 ts_tablet_manager.cc:1918] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:25.670642 25997 log.cc:1196] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/619ea29970f646c6933d5be3ece828b4
I20260811 18:48:25.670945 25997 ts_tablet_manager.cc:1939] T 619ea29970f646c6933d5be3ece828b4 P 32a2ef433ba84969a5e9ed940f134576: Deleting consensus metadata
I20260811 18:48:25.671391 24910 catalog_manager.cc:4841] TS 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189): tablet 619ea29970f646c6933d5be3ece828b4 (table test-table [id=91289a6cc529414db390a2c82cbf420d]) successfully deleted
I20260811 18:48:25.724925 25953 raft_consensus.cc:482] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:25.725072 25953 raft_consensus.cc:504] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.725457 25953 leader_election.cc:288] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189)
I20260811 18:48:25.729087 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.725785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:48531 (local address 127.9.215.3:41189)
0811 18:48:25.725834 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.725836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.725847 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:25.725957 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.725959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.725969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.725997 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.726232 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.726978 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.728196 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.728740 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.728820 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.728833 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.728970 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.729016 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:48:25.729110 26009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.725657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:41189 (local address 127.9.215.2:48531)
0811 18:48:25.725789 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.725802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.725858 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.726086 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.726086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.726088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.726145 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.726147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.727107 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.727109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.728083 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.728086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.728817 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.728818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.728895 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.728899 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.728950 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.728995 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260811 18:48:25.729682 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.725888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:58211 (local address 127.9.215.1:35773)
0811 18:48:25.725928 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.725930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.725937 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:25.725965 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.725966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.725974 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.726052 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:25.727106 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.727798 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.728776 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.729292 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.729359 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.729370 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.729568 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.729613 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:48:25.729676 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf06cead018b4595a0b7c8304f6c41e2" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a2ef433ba84969a5e9ed940f134576" is_pre_election: true
I20260811 18:48:25.729777 25810 raft_consensus.cc:2454] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 0.
I20260811 18:48:25.729947 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.725657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:35773 (local address 127.9.215.2:58211)
0811 18:48:25.725851 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.725860 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.725902 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.726062 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.726063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.726065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:25.726143 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.726145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.727904 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.727905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.728686 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.728688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.729371 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.729372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.729453 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.729458 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.729811 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.729861 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260811 18:48:25.729975 25374 leader_election.cc:302] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, 7253e32dbdf34f02926a609942214d86; no voters:
I20260811 18:48:25.730145 25953 raft_consensus.cc:2781] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:25.730194 25953 raft_consensus.cc:482] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:25.730226 25953 raft_consensus.cc:3037] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.730350 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf06cead018b4595a0b7c8304f6c41e2" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071" is_pre_election: true
I20260811 18:48:25.730481 25214 raft_consensus.cc:2454] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 0.
I20260811 18:48:25.731433 25953 raft_consensus.cc:504] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.731657 25953 leader_election.cc:288] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 election: Requested vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189)
I20260811 18:48:25.731859 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf06cead018b4595a0b7c8304f6c41e2" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071"
I20260811 18:48:25.731940 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf06cead018b4595a0b7c8304f6c41e2" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a2ef433ba84969a5e9ed940f134576"
I20260811 18:48:25.731974 25214 raft_consensus.cc:3037] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.732033 25810 raft_consensus.cc:3037] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.732689 25214 raft_consensus.cc:2454] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 1.
I20260811 18:48:25.732882 25810 raft_consensus.cc:2454] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 1.
I20260811 18:48:25.732878 25366 leader_election.cc:302] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7253e32dbdf34f02926a609942214d86, fc71383dae124ee0bf1b6b0215d97071; no voters:
I20260811 18:48:25.732982 25953 raft_consensus.cc:2781] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:25.733129 25953 raft_consensus.cc:686] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 1 LEADER]: Becoming Leader. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Running, Role: LEADER
I20260811 18:48:25.733244 25953 consensus_queue.cc:237] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.734050 24928 catalog_manager.cc:5495] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7253e32dbdf34f02926a609942214d86 (127.9.215.2). New cstate: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:48:25 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474104, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.62@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:25.776055 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.768413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39858 (local address 127.9.215.62:37445)
0811 18:48:25.768475 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:25.768478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:25.768609 (+ 131us) server_negotiation.cc:407] Connection header received
0811 18:48:25.768671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:25.768671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:25.768682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:25.768728 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:25.769191 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.769683 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.770371 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.770550 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.770604 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.773078 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:25.773080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:25.773081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:25.773088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:25.774557 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.775027 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.775028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.775028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.775105 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:25.775328 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:25.775330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:25.775330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:25.775551 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:25.775614 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:25.775815 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 18:48:25.775940 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":35}
I20260811 18:48:25.776059 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:25.768327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37445 (local address 127.0.0.1:39858)
0811 18:48:25.768523 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:25.768535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:25.768616 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:25.768738 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:25.768738 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:25.769060 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:25.769109 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.769111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.769751 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.769752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.770293 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:25.770294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:25.770607 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:25.770608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:25.770658 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:25.771304 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:25.771311 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:25.772951 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:25.774648 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.774658 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.774660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.774936 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.775169 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:25.775170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:25.775171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:25.775233 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:25.775617 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:25.775625 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:25.775687 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 18:48:25.775750 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:48:25.775882 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260811 18:48:25.778205 24928 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:39858:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:48:25.778453 24928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:25.779974 25173 tablet_service.cc:1467] Processing CreateTablet for tablet a7cbff11c0754abca0eab4fda1c7c699 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e235c421fa1419bb6780b1240efad5d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:48:25.780061 25762 tablet_service.cc:1467] Processing CreateTablet for tablet a7cbff11c0754abca0eab4fda1c7c699 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e235c421fa1419bb6780b1240efad5d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:48:25.780103 25474 tablet_service.cc:1467] Processing CreateTablet for tablet a7cbff11c0754abca0eab4fda1c7c699 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e235c421fa1419bb6780b1240efad5d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:48:25.780159 25173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7cbff11c0754abca0eab4fda1c7c699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.780192 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7cbff11c0754abca0eab4fda1c7c699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.780267 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7cbff11c0754abca0eab4fda1c7c699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:25.781725 25942 tablet_bootstrap.cc:492] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: Bootstrap starting.
I20260811 18:48:25.781956 25941 tablet_bootstrap.cc:492] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: Bootstrap starting.
I20260811 18:48:25.782151 25942 tablet_bootstrap.cc:654] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.782390 25941 tablet_bootstrap.cc:654] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.782753 25948 tablet_bootstrap.cc:492] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: Bootstrap starting.
I20260811 18:48:25.782796 25942 tablet_bootstrap.cc:492] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: No bootstrap required, opened a new log
I20260811 18:48:25.782830 25942 ts_tablet_manager.cc:1397] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:25.782987 25942 raft_consensus.cc:348] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.783041 25948 tablet_bootstrap.cc:654] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:25.783056 25942 raft_consensus.cc:374] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.783074 25942 raft_consensus.cc:729] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.783124 25942 consensus_queue.cc:260] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.783224 25942 ts_tablet_manager.cc:1428] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.783650 25941 tablet_bootstrap.cc:492] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: No bootstrap required, opened a new log
I20260811 18:48:25.783598 25954 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:25.783721 25941 ts_tablet_manager.cc:1397] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:25.783713 25954 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:25.783878 25941 raft_consensus.cc:348] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.783964 25941 raft_consensus.cc:374] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.783993 25941 raft_consensus.cc:729] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.784032 25941 consensus_queue.cc:260] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.784134 25941 ts_tablet_manager.cc:1428] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.784235 25953 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:25.784291 25948 tablet_bootstrap.cc:492] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: No bootstrap required, opened a new log
I20260811 18:48:25.784309 25953 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:25.784337 25948 ts_tablet_manager.cc:1397] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:48:25.784442 25948 raft_consensus.cc:348] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.784502 25948 raft_consensus.cc:374] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:25.784518 25948 raft_consensus.cc:729] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71383dae124ee0bf1b6b0215d97071, State: Initialized, Role: FOLLOWER
I20260811 18:48:25.784569 25948 consensus_queue.cc:260] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.784648 25948 ts_tablet_manager.cc:1428] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:25.784729 25981 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:25.784832 25981 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:25.997159 25953 raft_consensus.cc:482] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:25.997283 25953 raft_consensus.cc:504] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.997454 25953 leader_election.cc:288] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189)
I20260811 18:48:25.997720 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7cbff11c0754abca0eab4fda1c7c699" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a2ef433ba84969a5e9ed940f134576" is_pre_election: true
I20260811 18:48:25.997819 25810 raft_consensus.cc:2454] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 0.
I20260811 18:48:25.997753 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7cbff11c0754abca0eab4fda1c7c699" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071" is_pre_election: true
I20260811 18:48:25.997869 25214 raft_consensus.cc:2454] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 0.
I20260811 18:48:25.997977 25374 leader_election.cc:302] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, 7253e32dbdf34f02926a609942214d86; no voters:
I20260811 18:48:25.998102 25953 raft_consensus.cc:2781] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:25.998149 25953 raft_consensus.cc:482] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:25.998176 25953 raft_consensus.cc:3037] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.998867 25953 raft_consensus.cc:504] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:25.999017 25953 leader_election.cc:288] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 election: Requested vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189)
I20260811 18:48:25.999168 25810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7cbff11c0754abca0eab4fda1c7c699" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a2ef433ba84969a5e9ed940f134576"
I20260811 18:48:25.999178 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7cbff11c0754abca0eab4fda1c7c699" candidate_uuid: "7253e32dbdf34f02926a609942214d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071"
I20260811 18:48:25.999236 25810 raft_consensus.cc:3037] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.999249 25214 raft_consensus.cc:3037] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:25.999873 25810 raft_consensus.cc:2454] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 1.
I20260811 18:48:25.999975 25214 raft_consensus.cc:2454] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7253e32dbdf34f02926a609942214d86 in term 1.
I20260811 18:48:26.000084 25374 leader_election.cc:302] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, 7253e32dbdf34f02926a609942214d86; no voters:
I20260811 18:48:26.000191 25953 raft_consensus.cc:2781] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:26.000260 25953 raft_consensus.cc:686] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [term 1 LEADER]: Becoming Leader. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Running, Role: LEADER
I20260811 18:48:26.000329 25953 consensus_queue.cc:237] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.000674 26011 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: TxnStatusTablet state changed. Reason: New leader 7253e32dbdf34f02926a609942214d86. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.000744 26011 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:48:26.000924 26031 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:48:26.001040 26031 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:48:26.001123 24928 catalog_manager.cc:5495] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7253e32dbdf34f02926a609942214d86 (127.9.215.2). New cstate: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } health_report { overall_health: UNKNOWN } } }
I20260811 18:48:26.016268 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.012634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:41439 (local address 127.0.0.1:43688)
0811 18:48:26.012742 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.012753 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:26.012801 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:26.013245 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:26.013246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:26.013247 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:48:26.013303 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.013306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.014393 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.014394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.015149 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.015154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.015651 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.015652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.015720 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.015773 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.016096 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.016107 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:48:26.016142 (+ 35us) client_negotiation.cc:240] Negotiation successful
0811 18:48:26.016198 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 18:48:26.016269 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.012904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.9.215.2:41439)
0811 18:48:26.012967 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.012969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.012979 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:26.013039 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.013040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.013064 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.013154 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:26.013456 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.014282 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.015239 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.015570 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.015644 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.015877 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.016030 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.016107 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.016149 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.016175 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:48:26.017616 25214 raft_consensus.cc:1261] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Refusing update from remote peer 7253e32dbdf34f02926a609942214d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.017877 25810 raft_consensus.cc:1261] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Refusing update from remote peer 7253e32dbdf34f02926a609942214d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.017975 25953 consensus_queue.cc:1035] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.018208 25953 consensus_queue.cc:1035] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.018692 25981 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: TxnStatusTablet state changed. Reason: New leader 7253e32dbdf34f02926a609942214d86. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.018806 25981 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:26.018842 25954 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: TxnStatusTablet state changed. Reason: New leader 7253e32dbdf34f02926a609942214d86. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.018934 25954 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:26.019336 25953 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.019430 25953 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:48:26.019876 26011 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.019959 26011 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 7253e32dbdf34f02926a609942214d86: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:48:26.020279 25977 mvcc.cc:205] Tried to move back new op lower bound from 7317397938245517312 to 7317397938177712128. Current Snapshot: MvccSnapshot[applied={T|T < 7317397938245517312}]
I20260811 18:48:26.020411 25954 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.020481 25954 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P 32a2ef433ba84969a5e9ed940f134576: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:26.021008 25970 mvcc.cc:205] Tried to move back new op lower bound from 7317397938245517312 to 7317397938177712128. Current Snapshot: MvccSnapshot[applied={T|T < 7317397938245517312}]
I20260811 18:48:26.021129 25981 tablet_replica.cc:439] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7253e32dbdf34f02926a609942214d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } } }
I20260811 18:48:26.021215 25981 tablet_replica.cc:442] T a7cbff11c0754abca0eab4fda1c7c699 P fc71383dae124ee0bf1b6b0215d97071: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:48:26.024442 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.021821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.9.215.2:41439)
0811 18:48:26.021857 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.021859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.021869 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:26.021956 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.021957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.021968 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.022025 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:26.022114 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.022894 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.023691 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.024027 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.024095 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.024213 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.024250 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.024302 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.024339 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.024359 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 18:48:26.030293 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.027432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:35503 (local address 127.9.215.2:41439)
0811 18:48:26.027498 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.027500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.027669 (+ 169us) server_negotiation.cc:407] Connection header received
0811 18:48:26.027711 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.027712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.027722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.027777 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:26.027896 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.028611 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.029430 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.029728 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.029802 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.030029 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.030064 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.030113 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.030155 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.030174 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:48:26.030303 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.027350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:41439 (local address 127.9.215.2:35503)
0811 18:48:26.027600 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.027608 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:26.027665 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:26.027779 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:26.027779 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:26.027780 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:48:26.027840 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.027841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.028705 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.028707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.029355 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.029358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.029830 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.029831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.029891 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.029910 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.030118 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.030119 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 18:48:26.030150 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 18:48:26.030190 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
I20260811 18:48:26.032754 25214 raft_consensus.cc:1261] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Refusing update from remote peer 7253e32dbdf34f02926a609942214d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.032779 25810 raft_consensus.cc:1261] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Refusing update from remote peer 7253e32dbdf34f02926a609942214d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.032989 26041 consensus_queue.cc:1035] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.033134 25953 consensus_queue.cc:1035] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.048875 24928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:26.048990 24928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:26.049666 24928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Sending DeleteTablet for 3 replicas of tablet bf06cead018b4595a0b7c8304f6c41e2
I20260811 18:48:26.049979 25762 tablet_service.cc:1514] Processing DeleteTablet for tablet bf06cead018b4595a0b7c8304f6c41e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43824
I20260811 18:48:26.049980 25173 tablet_service.cc:1514] Processing DeleteTablet for tablet bf06cead018b4595a0b7c8304f6c41e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36510
I20260811 18:48:26.050086 25997 tablet_replica.cc:330] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: stopping tablet replica
I20260811 18:48:26.050091 25996 tablet_replica.cc:330] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: stopping tablet replica
I20260811 18:48:26.050135 25997 raft_consensus.cc:2229] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:26.050145 25996 raft_consensus.cc:2229] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:26.050172 25997 raft_consensus.cc:2258] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.050185 25996 raft_consensus.cc:2258] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.050545 25997 ts_tablet_manager.cc:1905] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.050542 25996 ts_tablet_manager.cc:1905] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.052498 25996 ts_tablet_manager.cc:1918] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:26.052572 25996 log.cc:1196] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/bf06cead018b4595a0b7c8304f6c41e2
I20260811 18:48:26.052778 25996 ts_tablet_manager.cc:1939] T bf06cead018b4595a0b7c8304f6c41e2 P fc71383dae124ee0bf1b6b0215d97071: Deleting consensus metadata
I20260811 18:48:26.052948 25472 tablet_service.cc:1514] Processing DeleteTablet for tablet bf06cead018b4595a0b7c8304f6c41e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43684
I20260811 18:48:26.053365 24901 catalog_manager.cc:4841] TS fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773): tablet bf06cead018b4595a0b7c8304f6c41e2 (table test-table [id=9601f6104cc44cadae39dbd1fc3f8123]) successfully deleted
I20260811 18:48:26.053424 25993 tablet_replica.cc:330] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: stopping tablet replica
I20260811 18:48:26.053485 25993 raft_consensus.cc:2229] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:26.053611 25993 raft_consensus.cc:2258] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.053910 25997 ts_tablet_manager.cc:1918] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:26.053964 25997 log.cc:1196] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/bf06cead018b4595a0b7c8304f6c41e2
I20260811 18:48:26.054147 25997 ts_tablet_manager.cc:1939] T bf06cead018b4595a0b7c8304f6c41e2 P 32a2ef433ba84969a5e9ed940f134576: Deleting consensus metadata
I20260811 18:48:26.054505 25993 ts_tablet_manager.cc:1905] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.056180 25993 ts_tablet_manager.cc:1918] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:26.056241 25993 log.cc:1196] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/bf06cead018b4595a0b7c8304f6c41e2
I20260811 18:48:26.056432 25993 ts_tablet_manager.cc:1939] T bf06cead018b4595a0b7c8304f6c41e2 P 7253e32dbdf34f02926a609942214d86: Deleting consensus metadata
I20260811 18:48:26.056847 24910 catalog_manager.cc:4841] TS 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189): tablet bf06cead018b4595a0b7c8304f6c41e2 (table test-table [id=9601f6104cc44cadae39dbd1fc3f8123]) successfully deleted
I20260811 18:48:26.057044 24901 catalog_manager.cc:4841] TS 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439): tablet bf06cead018b4595a0b7c8304f6c41e2 (table test-table [id=9601f6104cc44cadae39dbd1fc3f8123]) successfully deleted
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[24840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:48:26.077306 25021 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:26.071206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.9.215.62:37445)
0811 18:48:26.071275 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.071278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.071433 (+ 155us) server_negotiation.cc:407] Connection header received
0811 18:48:26.071611 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.071613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.071627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.071680 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:26.072291 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.073065 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.073817 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.074114 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.074188 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.077234 (+ 3046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39866: BlockingRecv error: recv got EOF from 127.0.0.1:39866 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[24840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM
I20260811 18:48:26.085537 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.078170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.9.215.62:37445)
0811 18:48:26.078225 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.078227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.078245 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:48:26.078303 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.078304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.078315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.078369 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:26.078832 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.079330 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.080063 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.080276 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.080332 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.082815 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:26.082817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:26.082818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:26.082826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:26.084306 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:26.084738 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:26.084739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:26.084739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:26.084790 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:26.084985 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:26.084986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:26.084986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:26.085090 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:26.085155 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.085371 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.085449 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:48:26.087181 24928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:26.087354 24928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:26.088799 25173 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4c605980fe4d8a99aaf17471ab5a0d (DEFAULT_TABLE table=test-table [id=7fcad0841ae349bc952ecc7cf2e2dbec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:26.088810 25472 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4c605980fe4d8a99aaf17471ab5a0d (DEFAULT_TABLE table=test-table [id=7fcad0841ae349bc952ecc7cf2e2dbec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:26.088852 25762 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4c605980fe4d8a99aaf17471ab5a0d (DEFAULT_TABLE table=test-table [id=7fcad0841ae349bc952ecc7cf2e2dbec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:26.088960 25173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4c605980fe4d8a99aaf17471ab5a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:26.088970 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4c605980fe4d8a99aaf17471ab5a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:26.088971 25472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4c605980fe4d8a99aaf17471ab5a0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:26.090372 25942 tablet_bootstrap.cc:492] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Bootstrap starting.
I20260811 18:48:26.090418 25941 tablet_bootstrap.cc:492] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Bootstrap starting.
I20260811 18:48:26.090806 25942 tablet_bootstrap.cc:654] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:26.090806 25941 tablet_bootstrap.cc:654] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:26.091187 25948 tablet_bootstrap.cc:492] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Bootstrap starting.
I20260811 18:48:26.091602 25948 tablet_bootstrap.cc:654] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:26.091601 25942 tablet_bootstrap.cc:492] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: No bootstrap required, opened a new log
I20260811 18:48:26.091653 25942 ts_tablet_manager.cc:1397] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:26.091818 25942 raft_consensus.cc:348] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.091893 25942 raft_consensus.cc:374] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:26.091921 25942 raft_consensus.cc:729] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Initialized, Role: FOLLOWER
I20260811 18:48:26.091964 25942 consensus_queue.cc:260] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.092058 25942 ts_tablet_manager.cc:1428] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:26.092355 25948 tablet_bootstrap.cc:492] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: No bootstrap required, opened a new log
I20260811 18:48:26.092407 25948 ts_tablet_manager.cc:1397] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:26.092509 25948 raft_consensus.cc:348] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.092573 25948 raft_consensus.cc:374] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:26.092592 25948 raft_consensus.cc:729] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71383dae124ee0bf1b6b0215d97071, State: Initialized, Role: FOLLOWER
I20260811 18:48:26.092633 25948 consensus_queue.cc:260] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.092723 25948 ts_tablet_manager.cc:1428] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:26.093248 25941 tablet_bootstrap.cc:492] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: No bootstrap required, opened a new log
I20260811 18:48:26.093299 25941 ts_tablet_manager.cc:1397] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:48:26.093444 25941 raft_consensus.cc:348] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.093502 25941 raft_consensus.cc:374] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:26.093518 25941 raft_consensus.cc:729] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Initialized, Role: FOLLOWER
I20260811 18:48:26.093561 25941 consensus_queue.cc:260] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.093652 25941 ts_tablet_manager.cc:1428] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:26.223724 25954 raft_consensus.cc:482] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:26.223850 25954 raft_consensus.cc:504] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.224195 25954 leader_election.cc:288] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:26.228210 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.224785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:48717 (local address 127.9.215.2:41439)
0811 18:48:26.224840 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.224843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.224875 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:48:26.224905 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.224906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.224916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.224991 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:26.225192 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.225662 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.227391 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.227816 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.227881 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.227893 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.228074 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.228137 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:48:26.228214 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.224673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:41439 (local address 127.9.215.3:48717)
0811 18:48:26.224770 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.224780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:26.224828 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:26.225002 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:26.225003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:26.225006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:26.225093 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.225094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.225747 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.225748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.227280 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.227282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.227916 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.227917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.227971 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.227975 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:26.228054 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:48:26.228100 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":21,"thread_start_us":43,"threads_started":1}
I20260811 18:48:26.228286 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.224505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:53673 (local address 127.9.215.1:35773)
0811 18:48:26.224563 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.224565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.224669 (+ 104us) server_negotiation.cc:407] Connection header received
0811 18:48:26.224865 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.224867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.224883 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.224914 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:26.225165 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.225906 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.227366 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.227940 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.228014 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.228024 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.228176 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.228228 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:48:26.228529 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.224448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:35773 (local address 127.9.215.3:53673)
0811 18:48:26.224587 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.224598 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:26.224679 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:26.225002 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:26.225003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:26.225009 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:26.225082 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.225085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.226024 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.226025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.227252 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.227257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.228022 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.228023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.228090 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.228093 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:26.228404 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 18:48:26.228454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260811 18:48:26.228519 25516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4c605980fe4d8a99aaf17471ab5a0d" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7253e32dbdf34f02926a609942214d86" is_pre_election: true
I20260811 18:48:26.228606 25516 raft_consensus.cc:2454] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 0.
I20260811 18:48:26.228804 25663 leader_election.cc:302] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, 7253e32dbdf34f02926a609942214d86; no voters:
I20260811 18:48:26.228950 25954 raft_consensus.cc:2781] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:26.228931 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4c605980fe4d8a99aaf17471ab5a0d" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071" is_pre_election: true
I20260811 18:48:26.229001 25954 raft_consensus.cc:482] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:26.229022 25954 raft_consensus.cc:3037] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:26.229030 25214 raft_consensus.cc:2454] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 0.
I20260811 18:48:26.229893 25954 raft_consensus.cc:504] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.230039 25954 leader_election.cc:288] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 election: Requested vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:26.230268 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4c605980fe4d8a99aaf17471ab5a0d" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071"
I20260811 18:48:26.230283 25516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c4c605980fe4d8a99aaf17471ab5a0d" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7253e32dbdf34f02926a609942214d86"
I20260811 18:48:26.230360 25516 raft_consensus.cc:3037] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:26.230355 25214 raft_consensus.cc:3037] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:26.231022 25214 raft_consensus.cc:2454] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 1.
I20260811 18:48:26.231132 25516 raft_consensus.cc:2454] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 1.
I20260811 18:48:26.231208 25663 leader_election.cc:302] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, fc71383dae124ee0bf1b6b0215d97071; no voters:
I20260811 18:48:26.231309 25954 raft_consensus.cc:2781] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:26.231539 25954 raft_consensus.cc:686] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 1 LEADER]: Becoming Leader. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Running, Role: LEADER
I20260811 18:48:26.231630 25954 consensus_queue.cc:237] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.232414 24928 catalog_manager.cc:5495] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3). New cstate: current_term: 1 leader_uuid: "32a2ef433ba84969a5e9ed940f134576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } health_report { overall_health: HEALTHY } } }
I20260811 18:48:26.243510 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.240958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.9.215.3:41189)
0811 18:48:26.241148 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.241150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.241156 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:48:26.241197 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.241197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.241217 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.241298 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:26.241433 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.241944 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.242688 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.242905 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.242959 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.243124 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.243238 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.243314 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.243351 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.243375 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":121,"threads_started":1}
I20260811 18:48:26.244663 25214 raft_consensus.cc:1261] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Refusing update from remote peer 32a2ef433ba84969a5e9ed940f134576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.244669 25516 raft_consensus.cc:1261] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Refusing update from remote peer 32a2ef433ba84969a5e9ed940f134576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.244954 25954 consensus_queue.cc:1035] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.245138 25954 consensus_queue.cc:1035] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.248637 24927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:26.248759 24927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:26.249466 24927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Sending DeleteTablet for 3 replicas of tablet 7c4c605980fe4d8a99aaf17471ab5a0d
I20260811 18:48:26.249765 25173 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c4c605980fe4d8a99aaf17471ab5a0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36510
I20260811 18:48:26.249788 25472 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c4c605980fe4d8a99aaf17471ab5a0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43684
I20260811 18:48:26.249861 25996 tablet_replica.cc:330] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: stopping tablet replica
I20260811 18:48:26.249882 25993 tablet_replica.cc:330] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: stopping tablet replica
I20260811 18:48:26.249912 25996 raft_consensus.cc:2229] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:26.249933 25993 raft_consensus.cc:2229] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:26.249954 25996 raft_consensus.cc:2258] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.249975 25993 raft_consensus.cc:2258] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.250483 25996 ts_tablet_manager.cc:1905] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.250658 25762 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c4c605980fe4d8a99aaf17471ab5a0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43824
I20260811 18:48:26.250759 25997 tablet_replica.cc:330] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: stopping tablet replica
I20260811 18:48:26.250802 25997 raft_consensus.cc:2229] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:26.250890 25997 raft_consensus.cc:2258] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.250993 24927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:26.251142 24927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:26.252251 25996 ts_tablet_manager.cc:1918] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:26.252314 25996 log.cc:1196] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/7c4c605980fe4d8a99aaf17471ab5a0d
I20260811 18:48:26.252381 25993 ts_tablet_manager.cc:1905] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.252446 25472 tablet_service.cc:1467] Processing CreateTablet for tablet ee9bc454ff794dc6b4871dde4b747238 (DEFAULT_TABLE table=test-table [id=ff52dd16123746d4ba07534ea472c4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:26.252517 25996 ts_tablet_manager.cc:1939] T 7c4c605980fe4d8a99aaf17471ab5a0d P fc71383dae124ee0bf1b6b0215d97071: Deleting consensus metadata
I20260811 18:48:26.252547 25472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee9bc454ff794dc6b4871dde4b747238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:26.252874 25173 tablet_service.cc:1467] Processing CreateTablet for tablet ee9bc454ff794dc6b4871dde4b747238 (DEFAULT_TABLE table=test-table [id=ff52dd16123746d4ba07534ea472c4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:26.252975 25173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee9bc454ff794dc6b4871dde4b747238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:26.253862 25941 tablet_bootstrap.cc:492] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Bootstrap starting.
I20260811 18:48:26.253973 25993 ts_tablet_manager.cc:1918] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:26.254031 25993 log.cc:1196] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/7c4c605980fe4d8a99aaf17471ab5a0d
I20260811 18:48:26.254164 25941 tablet_bootstrap.cc:654] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:26.254244 25993 ts_tablet_manager.cc:1939] T 7c4c605980fe4d8a99aaf17471ab5a0d P 7253e32dbdf34f02926a609942214d86: Deleting consensus metadata
I20260811 18:48:26.254339 25948 tablet_bootstrap.cc:492] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: Bootstrap starting.
I20260811 18:48:26.254657 25948 tablet_bootstrap.cc:654] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:26.254721 25762 tablet_service.cc:1467] Processing CreateTablet for tablet ee9bc454ff794dc6b4871dde4b747238 (DEFAULT_TABLE table=test-table [id=ff52dd16123746d4ba07534ea472c4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:26.254850 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee9bc454ff794dc6b4871dde4b747238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:26.254945 25941 tablet_bootstrap.cc:492] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: No bootstrap required, opened a new log
I20260811 18:48:26.254951 24901 catalog_manager.cc:4841] TS 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439): tablet 7c4c605980fe4d8a99aaf17471ab5a0d (table test-table [id=7fcad0841ae349bc952ecc7cf2e2dbec]) successfully deleted
I20260811 18:48:26.254985 25941 ts_tablet_manager.cc:1397] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:26.255012 24901 catalog_manager.cc:4841] TS fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773): tablet 7c4c605980fe4d8a99aaf17471ab5a0d (table test-table [id=7fcad0841ae349bc952ecc7cf2e2dbec]) successfully deleted
I20260811 18:48:26.255100 25941 raft_consensus.cc:348] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.255167 25941 raft_consensus.cc:374] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:26.255182 25941 raft_consensus.cc:729] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7253e32dbdf34f02926a609942214d86, State: Initialized, Role: FOLLOWER
I20260811 18:48:26.255220 25941 consensus_queue.cc:260] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.255312 25941 ts_tablet_manager.cc:1428] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:26.256366 25942 tablet_bootstrap.cc:492] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Bootstrap starting.
I20260811 18:48:26.256633 25997 ts_tablet_manager.cc:1905] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.256807 25942 tablet_bootstrap.cc:654] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:26.258005 25942 tablet_bootstrap.cc:492] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: No bootstrap required, opened a new log
I20260811 18:48:26.258059 25942 ts_tablet_manager.cc:1397] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:48:26.258076 25948 tablet_bootstrap.cc:492] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: No bootstrap required, opened a new log
I20260811 18:48:26.258132 25948 ts_tablet_manager.cc:1397] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 18:48:26.258199 25942 raft_consensus.cc:348] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.258242 25948 raft_consensus.cc:348] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.258293 25942 raft_consensus.cc:374] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:26.258309 25948 raft_consensus.cc:374] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:26.258316 25942 raft_consensus.cc:729] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Initialized, Role: FOLLOWER
I20260811 18:48:26.258327 25948 raft_consensus.cc:729] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71383dae124ee0bf1b6b0215d97071, State: Initialized, Role: FOLLOWER
I20260811 18:48:26.258369 25948 consensus_queue.cc:260] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.258363 25942 consensus_queue.cc:260] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.258447 25948 ts_tablet_manager.cc:1428] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:26.258463 25942 ts_tablet_manager.cc:1428] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 18:48:26.259303 25997 ts_tablet_manager.cc:1918] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:26.259358 25997 log.cc:1196] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/7c4c605980fe4d8a99aaf17471ab5a0d
I20260811 18:48:26.259578 25997 ts_tablet_manager.cc:1939] T 7c4c605980fe4d8a99aaf17471ab5a0d P 32a2ef433ba84969a5e9ed940f134576: Deleting consensus metadata
I20260811 18:48:26.259884 24910 catalog_manager.cc:4841] TS 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189): tablet 7c4c605980fe4d8a99aaf17471ab5a0d (table test-table [id=7fcad0841ae349bc952ecc7cf2e2dbec]) successfully deleted
I20260811 18:48:26.655993 25954 raft_consensus.cc:482] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:26.656121 25954 raft_consensus.cc:504] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.656283 25954 leader_election.cc:288] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:26.656556 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee9bc454ff794dc6b4871dde4b747238" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071" is_pre_election: true
I20260811 18:48:26.656665 25214 raft_consensus.cc:2454] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 0.
I20260811 18:48:26.656589 25516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee9bc454ff794dc6b4871dde4b747238" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7253e32dbdf34f02926a609942214d86" is_pre_election: true
I20260811 18:48:26.656723 25516 raft_consensus.cc:2454] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 0.
I20260811 18:48:26.656920 25663 leader_election.cc:302] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, fc71383dae124ee0bf1b6b0215d97071; no voters:
I20260811 18:48:26.657040 25954 raft_consensus.cc:2781] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:26.657089 25954 raft_consensus.cc:482] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:26.657110 25954 raft_consensus.cc:3037] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:26.657810 25954 raft_consensus.cc:504] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.657933 25954 leader_election.cc:288] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 election: Requested vote from peers fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773), 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439)
I20260811 18:48:26.658123 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee9bc454ff794dc6b4871dde4b747238" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71383dae124ee0bf1b6b0215d97071"
I20260811 18:48:26.658198 25214 raft_consensus.cc:3037] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:26.658144 25516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee9bc454ff794dc6b4871dde4b747238" candidate_uuid: "32a2ef433ba84969a5e9ed940f134576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7253e32dbdf34f02926a609942214d86"
I20260811 18:48:26.658226 25516 raft_consensus.cc:3037] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:26.658799 25214 raft_consensus.cc:2454] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 1.
I20260811 18:48:26.658912 25516 raft_consensus.cc:2454] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a2ef433ba84969a5e9ed940f134576 in term 1.
I20260811 18:48:26.658983 25663 leader_election.cc:302] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a2ef433ba84969a5e9ed940f134576, fc71383dae124ee0bf1b6b0215d97071; no voters:
I20260811 18:48:26.659086 25954 raft_consensus.cc:2781] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:26.659154 25954 raft_consensus.cc:686] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 1 LEADER]: Becoming Leader. State: Replica: 32a2ef433ba84969a5e9ed940f134576, State: Running, Role: LEADER
I20260811 18:48:26.659197 25954 consensus_queue.cc:237] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } }
I20260811 18:48:26.659843 24928 catalog_manager.cc:5495] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3). New cstate: current_term: 1 leader_uuid: "32a2ef433ba84969a5e9ed940f134576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a2ef433ba84969a5e9ed940f134576" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 41189 } health_report { overall_health: HEALTHY } } }
I20260811 18:48:26.705013 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.702670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:51371 (local address 127.9.215.2:41439)
0811 18:48:26.702757 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.702763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.702789 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:48:26.702868 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.702869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.702880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.702934 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:26.703051 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.703586 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.704342 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.704546 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.704602 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.704769 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.704807 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.704849 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.704903 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.704929 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:48:26.705029 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.702549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:41439 (local address 127.9.215.3:51371)
0811 18:48:26.702725 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.702739 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:26.702808 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:26.702937 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:26.702938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:26.702940 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:48:26.702996 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.702998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.703660 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.703662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.704259 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.704262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.704609 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.704610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.704662 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.704680 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.704858 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.704860 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:48:26.704901 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 18:48:26.704953 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260811 18:48:26.707015 25516 raft_consensus.cc:1261] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Refusing update from remote peer 32a2ef433ba84969a5e9ed940f134576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.707041 25214 raft_consensus.cc:1261] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Refusing update from remote peer 32a2ef433ba84969a5e9ed940f134576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:26.707321 25954 consensus_queue.cc:1035] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7253e32dbdf34f02926a609942214d86" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 41439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.707480 26080 consensus_queue.cc:1035] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71383dae124ee0bf1b6b0215d97071" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:26.714807 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.712404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:36609 (local address 127.9.215.3:41189)
0811 18:48:26.712460 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:26.712463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:26.712563 (+ 100us) server_negotiation.cc:407] Connection header received
0811 18:48:26.712634 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:26.712635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:26.712648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:26.712691 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:26.712820 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.713298 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.714033 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.714326 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.714378 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.714551 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.714594 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.714650 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:26.714719 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 18:48:26.714745 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:26.714814 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.712359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:41189 (local address 127.9.215.2:36609)
0811 18:48:26.712503 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.712514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:26.712577 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:26.712711 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:26.712712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:26.712714 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:48:26.712765 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.712767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.713366 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.713367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.713966 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:26.713968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:26.714389 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:26.714390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:26.714452 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:26.714457 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:48:26.714658 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:48:26.714661 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:26.714700 (+ 39us) client_negotiation.cc:240] Negotiation successful
0811 18:48:26.714747 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":99,"threads_started":1}
I20260811 18:48:26.720595 24927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:26.720705 24927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:26.721335 24927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4ce08a6f5474b3e8c4959fd23c9f219: Sending DeleteTablet for 3 replicas of tablet ee9bc454ff794dc6b4871dde4b747238
I20260811 18:48:26.721578 25173 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9bc454ff794dc6b4871dde4b747238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36510
I20260811 18:48:26.721643 25472 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9bc454ff794dc6b4871dde4b747238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43684
I20260811 18:48:26.721688 25762 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9bc454ff794dc6b4871dde4b747238 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:26 UTC) from {username='oryx', principal='oryx/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:43824
I20260811 18:48:26.721747 25996 tablet_replica.cc:330] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: stopping tablet replica
I20260811 18:48:26.721792 25997 tablet_replica.cc:330] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: stopping tablet replica
I20260811 18:48:26.721793 25993 tablet_replica.cc:330] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: stopping tablet replica
I20260811 18:48:26.721817 25996 raft_consensus.cc:2229] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:26.721853 25996 raft_consensus.cc:2258] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.721853 25993 raft_consensus.cc:2229] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:26.721859 25997 raft_consensus.cc:2229] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:48:26.721885 25993 raft_consensus.cc:2258] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.721920 25997 raft_consensus.cc:2258] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:26.722100 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 25069
I20260811 18:48:26.722332 25996 ts_tablet_manager.cc:1905] T ee9bc454ff794dc6b4871dde4b747238 P fc71383dae124ee0bf1b6b0215d97071: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.738270 25997 ts_tablet_manager.cc:1905] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.742913 25997 ts_tablet_manager.cc:1918] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:26.742993 25997 log.cc:1196] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/ee9bc454ff794dc6b4871dde4b747238
I20260811 18:48:26.743170 25997 ts_tablet_manager.cc:1939] T ee9bc454ff794dc6b4871dde4b747238 P 32a2ef433ba84969a5e9ed940f134576: Deleting consensus metadata
I20260811 18:48:26.738054 25993 ts_tablet_manager.cc:1905] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:26.743893 24910 catalog_manager.cc:4841] TS 32a2ef433ba84969a5e9ed940f134576 (127.9.215.3:41189): tablet ee9bc454ff794dc6b4871dde4b747238 (table test-table [id=ff52dd16123746d4ba07534ea472c4c2]) successfully deleted
I20260811 18:48:26.744719 25993 ts_tablet_manager.cc:1918] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:48:26.744796 25993 log.cc:1196] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/ee9bc454ff794dc6b4871dde4b747238
I20260811 18:48:26.744994 25993 ts_tablet_manager.cc:1939] T ee9bc454ff794dc6b4871dde4b747238 P 7253e32dbdf34f02926a609942214d86: Deleting consensus metadata
I20260811 18:48:26.745397 24901 catalog_manager.cc:4841] TS 7253e32dbdf34f02926a609942214d86 (127.9.215.2:41439): tablet ee9bc454ff794dc6b4871dde4b747238 (table test-table [id=ff52dd16123746d4ba07534ea472c4c2]) successfully deleted
W20260811 18:48:26.747602 24901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:35773 (error 108)
I20260811 18:48:26.747828 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 25326
I20260811 18:48:26.753845 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:26.747926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:35773 (local address 127.0.0.1:36526)
0811 18:48:26.753705 (+ 5779us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:26.753785 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:35773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5758,"thread_start_us":5628,"threads_started":1}
W20260811 18:48:26.753979 24901 catalog_manager.cc:4584] TS fc71383dae124ee0bf1b6b0215d97071 (127.9.215.1:35773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee9bc454ff794dc6b4871dde4b747238: Network error: Client connection negotiation failed: client connection to 127.9.215.1:35773: connect: Connection refused (error 111)
I20260811 18:48:26.774119 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 25624
I20260811 18:48:26.802122 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 24867
2026-08-11T18:48:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3529 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[26142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[26142](info): set up 2 sockets
Aug 11 18:48:26 dist-test-slave-pww0 krb5kdc[26142](info): commencing operation
krb5kdc: starting...
W20260811 18:48:28.052153 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:28 dist-test-slave-pww0 krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:28.187335 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:38355
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:38355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:28.272086 26167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:28.272329 26167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:28.272353 26167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:28.273609 26167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:28.273660 26167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:28.273674 26167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:28.273685 26167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:28.273696 26167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:28.275220 26167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:38355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:38355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:28.275445 26167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:28.275660 26167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:28.280716 26174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:28.280716 26191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:28.280771 26184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:28.280970 26167 server_base.cc:1034] running on GCE node
I20260811 18:48:28.281229 26167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:28.281533 26167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:28.282687 26167 hybrid_clock.cc:648] HybridClock initialized: now 1786474108282653 us; error 47 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T18:48:28Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1462 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:28 dist-test-slave-pww0 krb5kdc[26199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:28 dist-test-slave-pww0 krb5kdc[26199](info): set up 2 sockets
Aug 11 18:48:28 dist-test-slave-pww0 krb5kdc[26199](info): commencing operation
krb5kdc: starting...
W20260811 18:48:29.506441 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:29 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:29.635161 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35437
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:29.720175 26231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:29.720410 26231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:29.720433 26231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:29.721678 26231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:29.721727 26231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:29.721740 26231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:29.721751 26231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:29.721762 26231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:29.723363 26231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:29.723670 26231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:29.723948 26231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:29.729063 26242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:29.729055 26246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:29.729059 26240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:29.729143 26231 server_base.cc:1034] running on GCE node
I20260811 18:48:29.729503 26231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:29.729815 26231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:29.730989 26231 hybrid_clock.cc:648] HybridClock initialized: now 1786474109730961 us; error 34 us; skew 500 ppm
Aug 11 18:48:29 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:29.733676 26231 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:29.734192 26231 webserver.cc:460] Webserver started at http://127.9.215.62:32817/ using document root <none> and password file <none>
I20260811 18:48:29.734442 26231 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:29.734495 26231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:29.734637 26231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:29.735833 26231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "fb62e23de16740a09408f9927295f3c8"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "d3022836b319cd0d79b53db84f0e188e"
server_key_iv: "a5f5cd168019a3e7f512c6d0915d6ee3"
server_key_version: "encryptionkey@0"
I20260811 18:48:29.736204 26231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "fb62e23de16740a09408f9927295f3c8"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "d3022836b319cd0d79b53db84f0e188e"
server_key_iv: "a5f5cd168019a3e7f512c6d0915d6ee3"
server_key_version: "encryptionkey@0"
I20260811 18:48:29.738035 26231 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:29.739042 26256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:29.739403 26231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:29.739552 26231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "fb62e23de16740a09408f9927295f3c8"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "d3022836b319cd0d79b53db84f0e188e"
server_key_iv: "a5f5cd168019a3e7f512c6d0915d6ee3"
server_key_version: "encryptionkey@0"
I20260811 18:48:29.739631 26231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:29.767434 26231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:29.769912 26231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:29.770088 26231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:29.774564 26231 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:35437
I20260811 18:48:29.774609 26369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:35437 every 8 connection(s)
I20260811 18:48:29.774987 26231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:29.776135 26372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:29.778779 26372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Bootstrap starting.
I20260811 18:48:29.779414 26372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:29.779788 26372 log.cc:824] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:29.780666 26372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: No bootstrap required, opened a new log
I20260811 18:48:29.782001 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 26231
I20260811 18:48:29.782130 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:29.782326 10076 external_mini_cluster.cc:1428] Setting key f928021c9933e727539f1792652432a4
I20260811 18:48:29.782740 26372 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:29.782943 26372 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:29.783038 26372 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb62e23de16740a09408f9927295f3c8, State: Initialized, Role: FOLLOWER
I20260811 18:48:29.783281 26372 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:29.783371 26372 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:29.783397 26372 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:29.783450 26372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:29.784576 26372 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:29.784699 26372 leader_election.cc:302] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb62e23de16740a09408f9927295f3c8; no voters:
I20260811 18:48:29.784878 26372 leader_election.cc:288] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:29.784919 26379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:29.785058 26379 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 LEADER]: Becoming Leader. State: Replica: fb62e23de16740a09408f9927295f3c8, State: Running, Role: LEADER
I20260811 18:48:29.785164 26372 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:29.785221 26379 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:29.785984 26385 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb62e23de16740a09408f9927295f3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } } }
I20260811 18:48:29.785996 26383 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb62e23de16740a09408f9927295f3c8. Latest consensus state: current_term: 1 leader_uuid: "fb62e23de16740a09408f9927295f3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } } }
I20260811 18:48:29.786196 26385 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:29.786207 26383 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:29.786665 26403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:29.787227 26403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:29.789084 26403 catalog_manager.cc:1347] Generated new cluster ID: 92c08ce43861419ea1206d1bfc48a92e
I20260811 18:48:29.789145 26403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:48:29 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:29.792956 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:29.782956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.9.215.62:35437)
0811 18:48:29.783159 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:29.783168 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:29.783201 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:48:29.783321 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:29.783328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:29.783379 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:29.783550 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:29.784202 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:29.785355 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:29.786449 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:29.787112 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:29.787216 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:29.789799 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:29.789804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:29.789804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:29.789817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:29.791442 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:29.791793 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:29.791796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:29.791797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:29.791938 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:29.792129 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:29.792130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:29.792131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:29.792376 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:29.792530 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:29.792613 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 18:48:29.792756 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260811 18:48:29.803565 26403 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:29.804387 26403 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:29.808826 26403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Generated new TSK 0
I20260811 18:48:29.809041 26403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:29.842974 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35437
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:29.961292 26424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:29.961539 26424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:29.961705 26424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:29.963111 26424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:29.963171 26424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:29.963203 26424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:29.964967 26424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:29.965255 26424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:29.965517 26424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:29.971041 26441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:29.971055 26435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:29.971167 26436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:29.971398 26424 server_base.cc:1034] running on GCE node
I20260811 18:48:29.971640 26424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:29.971910 26424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:29.973083 26424 hybrid_clock.cc:648] HybridClock initialized: now 1786474109973067 us; error 33 us; skew 500 ppm
Aug 11 18:48:29 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:29.976385 26424 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:29.976992 26424 webserver.cc:460] Webserver started at http://127.9.215.1:41183/ using document root <none> and password file <none>
I20260811 18:48:29.977345 26424 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:29.977416 26424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:29.977545 26424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:29.978696 26424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "425f810a3fef419bb2392a8153b05349"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "269f6a6f332d4f34b28d80c0dfb0f7db"
server_key_iv: "4526f12ba1016a212e83272245148342"
server_key_version: "encryptionkey@0"
I20260811 18:48:29.979045 26424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "425f810a3fef419bb2392a8153b05349"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "269f6a6f332d4f34b28d80c0dfb0f7db"
server_key_iv: "4526f12ba1016a212e83272245148342"
server_key_version: "encryptionkey@0"
I20260811 18:48:29.980545 26424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:29.981428 26451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:29.981647 26424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:29.981751 26424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "425f810a3fef419bb2392a8153b05349"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "269f6a6f332d4f34b28d80c0dfb0f7db"
server_key_iv: "4526f12ba1016a212e83272245148342"
server_key_version: "encryptionkey@0"
I20260811 18:48:29.981827 26424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:29.992595 26424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:29.995008 26424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:29.995178 26424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:29.995450 26424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:29.995838 26424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:29.995874 26424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:29.995904 26424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:29.995929 26424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.001920 26424 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:37129
I20260811 18:48:30.001986 26672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:37129 every 8 connection(s)
I20260811 18:48:30.002321 26424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:30.010591 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 26424
I20260811 18:48:30.010706 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:30.010902 10076 external_mini_cluster.cc:1428] Setting key 0cb540451907651e98a7aaeaf59addf1
I20260811 18:48:30.012552 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.003067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:49685 (local address 127.9.215.62:35437)
0811 18:48:30.003143 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.003146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.003337 (+ 191us) server_negotiation.cc:407] Connection header received
0811 18:48:30.003944 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.003945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.003959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.004014 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.005023 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.005538 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.006565 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.006769 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.006827 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.009086 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:30.009088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:30.009088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:30.009117 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:30.010842 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:30.011339 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:30.011340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:30.011341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:30.011407 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:30.011774 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:30.011776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:30.011776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:30.011923 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:30.012000 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:30.012349 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 18:48:30.012459 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:48:30.012641 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.003040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:49685)
0811 18:48:30.003201 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.003246 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.003834 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.004104 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.004109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.004707 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.004917 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.004922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.005634 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.005635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.006468 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.006470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.006845 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.006846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.006984 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.007503 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.007512 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.008972 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:30.010930 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:30.010942 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:30.010945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:30.011273 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:30.011534 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:30.011535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:30.011536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:30.011703 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:30.012028 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:30.012036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:30.012194 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 18:48:30.012293 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 18:48:30.012439 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260811 18:48:30.013235 26677 heartbeater.cc:347] Connected to a master server at 127.9.215.62:35437
I20260811 18:48:30.013363 26677 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:30.013600 26677 heartbeater.cc:511] Master 127.9.215.62:35437 requested a full tablet report, sending...
I20260811 18:48:30.014293 26291 ts_manager.cc:194] Registered new tserver with Master: 425f810a3fef419bb2392a8153b05349 (127.9.215.1:37129)
I20260811 18:48:30.015106 26291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:49685
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:30.038725 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35437
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:30.126209 26688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:30.126430 26688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:30.126464 26688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:30.127719 26688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:30.127766 26688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:30.127803 26688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:30.129333 26688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:30.129578 26688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:30.129807 26688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:30.135046 26701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.135078 26706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.135046 26700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.135474 26688 server_base.cc:1034] running on GCE node
I20260811 18:48:30.135707 26688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:30.135980 26688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:30.137148 26688 hybrid_clock.cc:648] HybridClock initialized: now 1786474110137130 us; error 31 us; skew 500 ppm
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:30.139825 26688 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:30.140287 26688 webserver.cc:460] Webserver started at http://127.9.215.2:41465/ using document root <none> and password file <none>
I20260811 18:48:30.140475 26688 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:30.140517 26688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:30.140616 26688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:30.141696 26688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "2cba2ddbe0604a2ea1cf456139141461"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "e129a2ea5d509cc33eeb7df9f7d6a8e3"
server_key_iv: "cc1dc59d284d6a3e9480fc3adfdf667f"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.141993 26688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "2cba2ddbe0604a2ea1cf456139141461"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "e129a2ea5d509cc33eeb7df9f7d6a8e3"
server_key_iv: "cc1dc59d284d6a3e9480fc3adfdf667f"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.143378 26688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:48:30.144212 26723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.144447 26688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.144544 26688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "2cba2ddbe0604a2ea1cf456139141461"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "e129a2ea5d509cc33eeb7df9f7d6a8e3"
server_key_iv: "cc1dc59d284d6a3e9480fc3adfdf667f"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.144619 26688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:30.155843 26688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:30.158041 26688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:30.158198 26688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:30.158453 26688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:30.158772 26688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:30.158808 26688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.158838 26688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:30.158864 26688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.164724 26688 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42141
I20260811 18:48:30.164806 26950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42141 every 8 connection(s)
I20260811 18:48:30.165109 26688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:30.174813 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.166058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:37227 (local address 127.9.215.62:35437)
0811 18:48:30.166194 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.166197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.166400 (+ 203us) server_negotiation.cc:407] Connection header received
0811 18:48:30.166933 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.166935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.166948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.166998 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.168035 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.168544 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.169592 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.169802 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.169860 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.172202 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:30.172204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:30.172205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:30.172213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:30.173454 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:30.173863 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:30.173864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:30.173865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:30.173917 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:30.174144 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:30.174145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:30.174146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:30.174271 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:30.174329 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:30.174606 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 18:48:30.174717 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:48:30.174983 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.165980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:37227)
0811 18:48:30.166259 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.166309 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.166834 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.167091 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.167096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.167718 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.167932 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.167938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.168646 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.168647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.169494 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.169498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.169880 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.169881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.170005 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.170544 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.170553 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.172088 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:30.173536 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:30.173545 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:30.173548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:30.173790 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:30.173973 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:30.173974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:30.173975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:30.174087 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:30.174327 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:30.174335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:30.174484 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 18:48:30.174577 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:48:30.174752 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":138,"threads_started":1}
I20260811 18:48:30.175319 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 26688
I20260811 18:48:30.175436 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:30.175632 26951 heartbeater.cc:347] Connected to a master server at 127.9.215.62:35437
I20260811 18:48:30.175663 10076 external_mini_cluster.cc:1428] Setting key cb0388c0777ab6e914c157d3ddfc82c9
I20260811 18:48:30.175762 26951 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:30.175966 26951 heartbeater.cc:511] Master 127.9.215.62:35437 requested a full tablet report, sending...
I20260811 18:48:30.176358 26291 ts_manager.cc:194] Registered new tserver with Master: 2cba2ddbe0604a2ea1cf456139141461 (127.9.215.2:42141)
I20260811 18:48:30.176723 26291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:37227
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:30.204722 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35437
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:30.293421 26957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:30.293663 26957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:30.293689 26957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:30.294914 26957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:30.294957 26957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:30.294984 26957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:30.296558 26957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:30.296799 26957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:30.296998 26957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:30.302175 26968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.302157 26964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.302157 26965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.302917 26957 server_base.cc:1034] running on GCE node
I20260811 18:48:30.303181 26957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:30.303449 26957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:30.304661 26957 hybrid_clock.cc:648] HybridClock initialized: now 1786474110304639 us; error 37 us; skew 500 ppm
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:30.307327 26957 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:30.307870 26957 webserver.cc:460] Webserver started at http://127.9.215.3:40735/ using document root <none> and password file <none>
I20260811 18:48:30.308087 26957 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:30.308139 26957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:30.308282 26957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:30.309425 26957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "1fc12dbca7d94890bff397625afd7300"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "ff47ac563eb202ea3ac47955237e254c"
server_key_iv: "2854f0bd4e3077cc07a75b70fdab3674"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.309787 26957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "1fc12dbca7d94890bff397625afd7300"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "ff47ac563eb202ea3ac47955237e254c"
server_key_iv: "2854f0bd4e3077cc07a75b70fdab3674"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.311353 26957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:30.312397 26975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.312633 26957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.312747 26957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "1fc12dbca7d94890bff397625afd7300"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "ff47ac563eb202ea3ac47955237e254c"
server_key_iv: "2854f0bd4e3077cc07a75b70fdab3674"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.312827 26957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:30.321031 26957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:30.323299 26957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:30.323451 26957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:30.323742 26957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:30.324069 26957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:30.324102 26957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.324132 26957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:30.324150 26957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.330446 26957 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44029
I20260811 18:48:30.330507 27212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44029 every 8 connection(s)
I20260811 18:48:30.330847 26957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:30.331135 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 26957
I20260811 18:48:30.331249 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:30.331485 10076 external_mini_cluster.cc:1428] Setting key d56d867c149828c010ee537f09540f66
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:30.340481 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.331704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:42705 (local address 127.9.215.62:35437)
0811 18:48:30.331766 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.331768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.332151 (+ 383us) server_negotiation.cc:407] Connection header received
0811 18:48:30.332691 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.332693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.332705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.332755 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.333751 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.334261 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.335289 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.335535 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.335596 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.337750 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:30.337752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:30.337752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:30.337760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:30.339088 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:30.339559 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:30.339560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:30.339560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:30.339614 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:30.339890 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:30.339890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:30.339890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:30.340004 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:30.340050 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:30.340331 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 18:48:30.340405 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:48:30.340667 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.331696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:42705)
0811 18:48:30.332014 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.332066 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.332587 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.332839 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.332844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.333441 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.333657 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.333663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.334351 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.334352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.335208 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.335211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.335613 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.335615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.335743 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.336224 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.336232 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.337651 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:30.339191 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:30.339201 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:30.339204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:30.339448 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:30.339698 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:30.339699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:30.339699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:30.339830 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:30.340053 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:30.340059 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:30.340206 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 18:48:30.340295 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:48:30.340437 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":90,"threads_started":1}
I20260811 18:48:30.341207 27214 heartbeater.cc:347] Connected to a master server at 127.9.215.62:35437
I20260811 18:48:30.341326 27214 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:30.341524 27214 heartbeater.cc:511] Master 127.9.215.62:35437 requested a full tablet report, sending...
I20260811 18:48:30.342036 26291 ts_manager.cc:194] Registered new tserver with Master: 1fc12dbca7d94890bff397625afd7300 (127.9.215.3:44029)
I20260811 18:48:30.342418 26291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:42705
I20260811 18:48:30.343355 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab.
Entry for principal oryx/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab.
WARNING: no policy specified for oryx/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab.
Entry for principal oryx/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab.
WARNING: no policy specified for oryx/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab.
Entry for principal oryx/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab.
I20260811 18:48:30.422525 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 26424
I20260811 18:48:30.444114 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 26688
I20260811 18:48:30.454126 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 26957
I20260811 18:48:30.467222 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 26231
I20260811 18:48:30.488445 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35437
--webserver_interface=127.9.215.62
--webserver_port=32817
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:30.573135 27223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:30.573360 27223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:30.573383 27223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:30.574599 27223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:30.574651 27223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:30.574666 27223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:30.574676 27223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:30.574692 27223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:30.576309 27223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=32817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:30.576563 27223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:30.576803 27223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:30.581830 27231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.581825 27236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.582110 27223 server_base.cc:1034] running on GCE node
W20260811 18:48:30.581835 27232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.582430 27223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:30.582755 27223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:30.583900 27223 hybrid_clock.cc:648] HybridClock initialized: now 1786474110583882 us; error 37 us; skew 500 ppm
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:30.586710 27223 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:30.587200 27223 webserver.cc:460] Webserver started at http://127.9.215.62:32817/ using document root <none> and password file <none>
I20260811 18:48:30.587430 27223 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:30.587529 27223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:30.588892 27223 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.589633 27251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.589840 27223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.589942 27223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "fb62e23de16740a09408f9927295f3c8"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "d3022836b319cd0d79b53db84f0e188e"
server_key_iv: "a5f5cd168019a3e7f512c6d0915d6ee3"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.590337 27223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:30.601648 27223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:30.604079 27223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:30.604255 27223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:30.608356 27223 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:35437
I20260811 18:48:30.608395 27355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:35437 every 8 connection(s)
I20260811 18:48:30.608781 27223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:30.610363 27357 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:48:30.611068 27357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Bootstrap starting.
I20260811 18:48:30.614109 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 27223
I20260811 18:48:30.614419 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:37129
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:35437
--webserver_port=41183
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:48:30.620800 27357 log.cc:824] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:30.622768 27357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:48:30.623077 27357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Bootstrap complete.
I20260811 18:48:30.625416 27357 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:30.625667 27357 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb62e23de16740a09408f9927295f3c8, State: Initialized, Role: FOLLOWER
I20260811 18:48:30.625824 27357 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:30.625897 27357 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:30.625943 27357 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:30.625996 27357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:48:30.627051 27357 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:30.627152 27357 leader_election.cc:302] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb62e23de16740a09408f9927295f3c8; no voters:
I20260811 18:48:30.627339 27357 leader_election.cc:288] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:48:30.627401 27367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:48:30.627643 27367 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [term 2 LEADER]: Becoming Leader. State: Replica: fb62e23de16740a09408f9927295f3c8, State: Running, Role: LEADER
I20260811 18:48:30.627671 27357 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:30.627812 27367 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } }
I20260811 18:48:30.628207 27369 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb62e23de16740a09408f9927295f3c8. Latest consensus state: current_term: 2 leader_uuid: "fb62e23de16740a09408f9927295f3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } } }
I20260811 18:48:30.628237 27368 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb62e23de16740a09408f9927295f3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb62e23de16740a09408f9927295f3c8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35437 } } }
I20260811 18:48:30.628306 27369 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:30.628321 27368 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:30.628504 27380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:30.629073 27380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:30.629305 27380 catalog_manager.cc:1259] Loaded cluster ID: 92c08ce43861419ea1206d1bfc48a92e
I20260811 18:48:30.629410 27380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:30.630606 27380 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:30.630789 27380 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb62e23de16740a09408f9927295f3c8: Loaded TSK: 0
I20260811 18:48:30.631000 27380 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:48:30.704082 27360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:30.704315 27360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:30.704341 27360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:30.705652 27360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:30.705709 27360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:30.705766 27360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:30.707520 27360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.1.keytab
--principal=oryx/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:37129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=41183
--tserver_master_addrs=127.9.215.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:30.707813 27360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:30.708062 27360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:30.713938 27418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.713954 27414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.713965 27413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.714648 27360 server_base.cc:1034] running on GCE node
I20260811 18:48:30.714892 27360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:30.715170 27360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:30.716351 27360 hybrid_clock.cc:648] HybridClock initialized: now 1786474110716334 us; error 32 us; skew 500 ppm
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:30.719225 27360 init.cc:377] Logged in from keytab as oryx/127.9.215.1@KRBTEST.COM (short username oryx)
I20260811 18:48:30.719784 27360 webserver.cc:460] Webserver started at http://127.9.215.1:41183/ using document root <none> and password file <none>
I20260811 18:48:30.720024 27360 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:30.720100 27360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:30.721662 27360 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.722337 27432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.722568 27360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.722679 27360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "425f810a3fef419bb2392a8153b05349"
format_stamp: "Formatted at 2026-08-11 18:48:29 on dist-test-slave-pww0"
server_key: "269f6a6f332d4f34b28d80c0dfb0f7db"
server_key_iv: "4526f12ba1016a212e83272245148342"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.723049 27360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:30.734186 27360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:30.736501 27360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:30.736650 27360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:30.736883 27360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:30.737192 27360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:30.737228 27360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.737259 27360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:30.737283 27360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.743085 27360 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:37129
I20260811 18:48:30.743146 27659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:37129 every 8 connection(s)
I20260811 18:48:30.743537 27360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:30.750550 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 27360
I20260811 18:48:30.750883 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:42141
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:35437
--webserver_port=41465
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:30.753639 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.744313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:42471)
0811 18:48:30.744513 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.744566 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.745282 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.745762 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.745766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.746538 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.746811 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.746818 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.748011 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.748012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.749379 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.749383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.750013 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.750014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.750157 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.750823 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.750834 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.753413 (+ 2579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
W20260811 18:48:30.753990 27660 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:35437 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.215.62:35437: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
W20260811 18:48:30.754012 27667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.744451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:42471 (local address 127.9.215.62:35437)
0811 18:48:30.744614 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.744623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.744659 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:48:30.745516 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.745524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.745571 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.745686 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.746933 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.747923 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.749485 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.749925 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.750008 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.753597 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:42471: BlockingRecv error: recv got EOF from 127.9.215.1:42471 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:30.760391 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.754273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:38669)
0811 18:48:30.754327 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.754337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.754373 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.754640 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.754641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.754786 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.754847 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.754849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.755818 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.755819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.756626 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.756630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.757103 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.757105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.757176 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.757717 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.757723 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.760306 (+ 2583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 18:48:30.760529 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.754311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:38669 (local address 127.9.215.62:35437)
0811 18:48:30.754484 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.754486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.754493 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:30.754518 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.754518 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.754528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.754596 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.754928 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.755723 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.756726 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.757014 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.757096 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.760449 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:38669: BlockingRecv error: recv got EOF from 127.9.215.1:38669 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:30.766813 27680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.760814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:59145)
0811 18:48:30.760993 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.761006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.761148 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.761302 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.761303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.761498 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.761582 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.761584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.762520 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.762522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.763448 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.763453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.764020 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.764021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.764098 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.764592 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.764597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.766712 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
W20260811 18:48:30.766897 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.761002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:59145 (local address 127.9.215.62:35437)
0811 18:48:30.761061 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.761063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.761142 (+ 79us) server_negotiation.cc:407] Connection header received
0811 18:48:30.761210 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.761211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.761224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.761283 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.761655 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.762429 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.763620 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.763910 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.763987 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.766834 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:59145: BlockingRecv error: recv got EOF from 127.9.215.1:59145 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 18:48:30.767220 27660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:48:30.838966 27674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:30.839210 27674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:30.839252 27674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:30.840528 27674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:30.840581 27674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:30.840619 27674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:30.842175 27674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.2.keytab
--principal=oryx/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:42141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=41465
--tserver_master_addrs=127.9.215.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:30.842437 27674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:30.842664 27674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:30.847698 27689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.847715 27688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.847877 27674 server_base.cc:1034] running on GCE node
W20260811 18:48:30.847764 27691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.848111 27674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:30.848378 27674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:30.849540 27674 hybrid_clock.cc:648] HybridClock initialized: now 1786474110849518 us; error 36 us; skew 500 ppm
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:30.852272 27674 init.cc:377] Logged in from keytab as oryx/127.9.215.2@KRBTEST.COM (short username oryx)
I20260811 18:48:30.852795 27674 webserver.cc:460] Webserver started at http://127.9.215.2:41465/ using document root <none> and password file <none>
I20260811 18:48:30.853036 27674 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:30.853106 27674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:30.854522 27674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.855291 27700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.855506 27674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.855592 27674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "2cba2ddbe0604a2ea1cf456139141461"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "e129a2ea5d509cc33eeb7df9f7d6a8e3"
server_key_iv: "cc1dc59d284d6a3e9480fc3adfdf667f"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.855969 27674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:30.865658 27674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:30.867841 27674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:30.867983 27674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:30.868233 27674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:30.868554 27674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:30.868589 27674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.868618 27674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:30.868635 27674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.874393 27674 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42141
I20260811 18:48:30.874439 27947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42141 every 8 connection(s)
I20260811 18:48:30.874790 27674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:30.877769 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 27674
I20260811 18:48:30.878041 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:44029
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:35437
--webserver_port=40735
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:30.882156 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.875504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:33351)
0811 18:48:30.875673 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.875715 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.876201 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.876403 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.876406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.876965 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.877182 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.877188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.877976 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.877977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.878827 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.878829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.879188 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.879189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.879318 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.879834 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.879842 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.882000 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260811 18:48:30.882200 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.875516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:33351 (local address 127.9.215.62:35437)
0811 18:48:30.875575 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.875577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.875796 (+ 219us) server_negotiation.cc:407] Connection header received
0811 18:48:30.876276 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.876277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.876290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.876337 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.877283 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.877823 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.878914 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.879115 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.879171 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.882131 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:33351: BlockingRecv error: recv got EOF from 127.9.215.2:33351 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260811 18:48:30.882628 27948 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:35437 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.215.62:35437: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:30.887512 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.882844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:34487)
0811 18:48:30.882886 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.882897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.882935 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.883085 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.883085 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.883204 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.883251 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.883252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.884158 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.884159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.884665 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.884667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.885142 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.885143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.885217 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.885658 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.885663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.887404 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 18:48:30.887624 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.882906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:34487 (local address 127.9.215.62:35437)
0811 18:48:30.882947 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.882949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.882978 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:48:30.883003 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.883004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.883016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.883082 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.883313 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.884065 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.884740 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.885034 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.885116 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.887555 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:34487: BlockingRecv error: recv got EOF from 127.9.215.2:34487 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:30.892036 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.887910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:45331)
0811 18:48:30.888007 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.888018 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.888065 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.888189 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.888190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.888323 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.888358 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.888359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.888978 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.888979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.889490 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.889492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.889808 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.889809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.889861 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.890276 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:30.890281 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:30.891964 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 18:48:30.892184 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.887965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:45331 (local address 127.9.215.62:35437)
0811 18:48:30.888011 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.888014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.888049 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:48:30.888124 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.888124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.888134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.888180 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.888421 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.888906 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.889563 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.889751 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.889803 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:30.892070 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:45331: BlockingRecv error: recv got EOF from 127.9.215.2:45331 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 18:48:30.892180 27948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:48:30.962798 27951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:30.963023 27951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:30.963048 27951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:30.964285 27951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:30.964335 27951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:30.964363 27951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:30.965875 27951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/krb5kdc/oryx_127.9.215.3.keytab
--principal=oryx/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:44029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=40735
--tserver_master_addrs=127.9.215.62:35437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:30.966116 27951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:30.966351 27951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:30.971349 27966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.971433 27962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:30.971441 27961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:30.971652 27951 server_base.cc:1034] running on GCE node
I20260811 18:48:30.971866 27951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:30.972100 27951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:30.973268 27951 hybrid_clock.cc:648] HybridClock initialized: now 1786474110973253 us; error 27 us; skew 500 ppm
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474110, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:30.975961 27951 init.cc:377] Logged in from keytab as oryx/127.9.215.3@KRBTEST.COM (short username oryx)
I20260811 18:48:30.976444 27951 webserver.cc:460] Webserver started at http://127.9.215.3:40735/ using document root <none> and password file <none>
I20260811 18:48:30.976660 27951 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:30.976729 27951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:30.978091 27951 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:30.978681 27976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.978828 27951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.978925 27951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "1fc12dbca7d94890bff397625afd7300"
format_stamp: "Formatted at 2026-08-11 18:48:30 on dist-test-slave-pww0"
server_key: "ff47ac563eb202ea3ac47955237e254c"
server_key_iv: "2854f0bd4e3077cc07a75b70fdab3674"
server_key_version: "encryptionkey@0"
I20260811 18:48:30.979305 27951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:30.986466 27951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:30.988376 27951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:30.988519 27951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:30.988741 27951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:30.989038 27951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:30.989073 27951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.989109 27951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:30.989133 27951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:30.995059 27951 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44029
I20260811 18:48:30.995119 28241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44029 every 8 connection(s)
I20260811 18:48:30.995483 27951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:30 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:48:31.002727 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:30.996220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:57869 (local address 127.9.215.62:35437)
0811 18:48:30.996265 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:30.996267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:30.996479 (+ 212us) server_negotiation.cc:407] Connection header received
0811 18:48:30.996997 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:30.996998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:30.997011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:30.997070 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:30.998020 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.998518 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.999618 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.999810 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.999864 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.002676 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:57869: BlockingRecv error: recv got EOF from 127.9.215.3:57869 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:31.002750 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:30.996196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:57869)
0811 18:48:30.996357 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:30.996396 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:30.996902 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:30.997142 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:30.997147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:30.997727 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:30.997925 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.997930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.998610 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.998612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:30.999531 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:30.999534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:30.999876 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:30.999878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.000014 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.000486 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:31.000495 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:31.002589 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
W20260811 18:48:31.002964 28244 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:35437 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.215.62:35437: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
I20260811 18:48:31.004117 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 27951
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:31.007802 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:31.003227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:44871)
0811 18:48:31.003275 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:31.003286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:31.003321 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:31.003600 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:31.003601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:31.003733 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:31.003782 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.003784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.004432 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.004433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.005074 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.005076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.005504 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.005505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.005562 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.005952 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:31.005957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:31.007749 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 18:48:31.007998 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:31.003429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:44871 (local address 127.9.215.62:35437)
0811 18:48:31.003499 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:31.003501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:31.003508 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:48:31.003528 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:31.003529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:31.003536 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:31.003587 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:31.003838 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.004368 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.005152 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.005436 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.005492 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.007937 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:44871: BlockingRecv error: recv got EOF from 127.9.215.3:44871 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:31.012398 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:31.004487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.9.215.62:35437)
0811 18:48:31.004540 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:31.004542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:31.004730 (+ 188us) server_negotiation.cc:407] Connection header received
0811 18:48:31.004770 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:31.004771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:31.004783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:31.004842 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:31.005462 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.006103 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.007085 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.007306 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.007356 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.008902 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:31.008908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:31.008908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:31.008920 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:31.010796 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:31.011195 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:31.011198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:31.011198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:31.011296 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:31.011547 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:31.011548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:31.011549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:31.011901 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:31.012017 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:31.012177 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:48:31.012302 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:48:31.012542 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:31.008032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:55425)
0811 18:48:31.008112 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:31.008121 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:31.008154 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:31.008290 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:31.008290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:31.008391 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:31.008418 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.008419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.009153 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.009154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.009839 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.009842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.010155 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.010156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.010214 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.010584 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:31.010589 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:31.012474 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 18:48:31.012786 28244 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:48:31.012995 27679 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:31.008115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:55425 (local address 127.9.215.62:35437)
0811 18:48:31.008150 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:31.008152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:31.008161 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:31.008217 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:31.008218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:31.008227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:31.008284 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:31.008478 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.009084 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.009913 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.010094 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.010141 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.012943 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:55425: BlockingRecv error: recv got EOF from 127.9.215.3:55425 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:31.772859 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:31.767726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:45535)
0811 18:48:31.767866 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:31.767906 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:31.768025 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:31.768236 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:31.768237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:31.768445 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:31.768501 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.768503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.769217 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.769218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.769757 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.769758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.770119 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.770120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.770178 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.770702 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:31.770708 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:31.772784 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
W20260811 18:48:31.772955 28251 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:31.767828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:45535 (local address 127.9.215.62:35437)
0811 18:48:31.767936 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:31.767939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:31.767997 (+ 58us) server_negotiation.cc:407] Connection header received
0811 18:48:31.768117 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:31.768118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:31.768131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:31.768217 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:31.768613 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.769140 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.769852 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.770056 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.770117 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.772883 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:45535: BlockingRecv error: recv got EOF from 127.9.215.1:45535 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:31 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:31.897804 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:31.892740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:60575)
0811 18:48:31.892887 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:31.892898 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:31.892958 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:31.893157 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:31.893158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:31.893354 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:31.893442 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.893445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.894122 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.894123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.894723 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:31.894726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.895145 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.895146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:31.895204 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.895753 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:31.895758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:31.897722 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
W20260811 18:48:31.897929 28251 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:31.892837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:60575 (local address 127.9.215.62:35437)
0811 18:48:31.892925 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:31.892927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:31.892940 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:48:31.893011 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:31.893012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:31.893027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:31.893094 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:31.893508 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.894007 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.894829 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:31.895046 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:31.895113 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:31.897864 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:60575: BlockingRecv error: recv got EOF from 127.9.215.2:60575 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 18:48:32 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:32 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:32.018179 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:32.013301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:42951)
0811 18:48:32.013438 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:32.013448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:32.013504 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:32.013693 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:32.013694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:32.013874 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:32.013931 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:32.013933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.014684 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.014686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:32.015404 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:32.015408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.015821 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.015822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:32.015868 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:32.016339 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:32.016344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:32.018103 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
W20260811 18:48:32.018339 28251 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:32.013390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:42951 (local address 127.9.215.62:35437)
0811 18:48:32.013474 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:32.013476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:32.013494 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:48:32.013561 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:32.013562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:32.013575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:32.013645 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:32.014047 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.014581 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.015533 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.015757 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.015815 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:32.018271 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:42951: BlockingRecv error: recv got EOF from 127.9.215.3:42951 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 18:48:32 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:32 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:32.778644 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:32.773435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:53339)
0811 18:48:32.773619 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:32.773630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:32.773705 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:32.773889 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:32.773890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:32.774054 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:32.774112 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:32.774113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.775015 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.775015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:32.775629 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:32.775632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.776039 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.776040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:32.776098 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:32.776601 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:32.776606 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:32.778570 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260811 18:48:32.778801 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:32.773582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:53339 (local address 127.9.215.62:35437)
0811 18:48:32.773743 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:32.773745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:32.773757 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:48:32.773784 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:32.773785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:32.773795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:32.773875 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:32.774210 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.774943 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.775747 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.775949 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.776005 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:32.778724 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:53339: BlockingRecv error: recv got EOF from 127.9.215.1:53339 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
Aug 11 18:48:32 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:32 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:32.903031 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:32.898365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:60473)
0811 18:48:32.898488 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:32.898499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:32.898576 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:32.898711 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:32.898712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:32.898877 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:32.898930 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:32.898931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.899763 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.899764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:32.900341 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:32.900344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.900702 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.900703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:32.900757 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:32.901201 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:32.901206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:32.902960 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
W20260811 18:48:32.903157 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:32.898461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:60473 (local address 127.9.215.62:35437)
0811 18:48:32.898541 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:32.898543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:32.898554 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:32.898631 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:32.898632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:32.898643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:32.898699 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:32.899033 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.899652 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.900410 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:32.900602 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:32.900663 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:32.903080 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:60473: BlockingRecv error: recv got EOF from 127.9.215.2:60473 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 18:48:33 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:33 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:33.023555 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:33.018769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:50235)
0811 18:48:33.018935 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:33.018946 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:33.019019 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:33.019272 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:33.019274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:33.019445 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:33.019533 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:33.019535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.020177 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.020179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:33.020725 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:33.020727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.021050 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.021052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:33.021109 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:33.021561 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:33.021566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:33.023442 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":64,"threads_started":1}
W20260811 18:48:33.023641 28260 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:33.018934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:50235 (local address 127.9.215.62:35437)
0811 18:48:33.018992 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:33.018994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:33.019003 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:33.019176 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:33.019177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:33.019193 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:33.019242 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:33.019606 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.020096 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.020803 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.020991 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.021045 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:33.023584 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:50235: BlockingRecv error: recv got EOF from 127.9.215.3:50235 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 11 18:48:33 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:33 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:33.784137 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:33.779174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:58953)
0811 18:48:33.779333 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:33.779343 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:33.779393 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:33.779564 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:33.779565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:33.779734 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:33.779785 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:33.779787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.780645 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.780646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:33.781199 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:33.781201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.781633 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.781635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:33.781688 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:33.782133 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:33.782138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:33.784067 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
W20260811 18:48:33.784291 28264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:33.779263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:58953 (local address 127.9.215.62:35437)
0811 18:48:33.779379 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:33.779381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:33.779392 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:33.779449 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:33.779450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:33.779483 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:33.779548 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:33.779879 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.780480 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.781324 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.781552 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.781616 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:33.784210 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:58953: BlockingRecv error: recv got EOF from 127.9.215.1:58953 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
Aug 11 18:48:33 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:33 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:33.909209 28265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:33.903686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:54689)
0811 18:48:33.903897 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:33.903908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:33.903984 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:33.904163 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:33.904164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:33.904323 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:33.904376 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:33.904378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.905343 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.905345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:33.905950 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:33.905953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.906469 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.906470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:33.906525 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:33.907156 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:33.907161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:33.909145 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260811 18:48:33.909473 28264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:33.903822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:54689 (local address 127.9.215.62:35437)
0811 18:48:33.903873 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:33.903875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:33.903994 (+ 119us) server_negotiation.cc:407] Connection header received
0811 18:48:33.904067 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:33.904068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:33.904085 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:33.904144 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:33.904445 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.905225 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.906051 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:33.906366 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:33.906457 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:33.909366 (+ 2909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:54689: BlockingRecv error: recv got EOF from 127.9.215.2:54689 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 18:48:34 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:34 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:34.029306 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:34.024110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:47981)
0811 18:48:34.024325 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:34.024339 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:34.024420 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:34.024560 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:34.024561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:34.024740 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:34.024787 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:34.024789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.025432 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.025433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:34.026036 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:34.026038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.026372 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.026374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:34.026430 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:34.026947 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:34.026953 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:34.029204 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":120,"threads_started":1}
W20260811 18:48:34.029376 28264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:34.024247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:47981 (local address 127.9.215.62:35437)
0811 18:48:34.024329 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:34.024331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:34.024405 (+ 74us) server_negotiation.cc:407] Connection header received
0811 18:48:34.024483 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:34.024484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:34.024498 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:34.024549 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:34.024856 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.025355 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.026111 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.026307 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.026364 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:34.029317 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:47981: BlockingRecv error: recv got EOF from 127.9.215.3:47981 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:48:34 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:34 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:34.793721 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:34.784696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:58049)
0811 18:48:34.784833 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:34.784843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:34.784895 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:34.785104 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:34.785109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:34.785280 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:34.785355 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:34.785357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.786121 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.786122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:34.786699 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:34.786701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.787078 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.787079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:34.787125 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:34.791808 (+ 4683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:34.791814 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:34.793654 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260811 18:48:34.793901 28272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:34.784780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:58049 (local address 127.9.215.62:35437)
0811 18:48:34.784913 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:34.784915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:34.784926 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:34.784953 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:34.784953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:34.784963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:34.785041 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:34.785459 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.786018 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.786798 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.787014 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.787077 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:34.793820 (+ 6743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:58049: BlockingRecv error: recv got EOF from 127.9.215.1:58049 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
Aug 11 18:48:34 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:34 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:34.914631 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:34.909823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:57303)
0811 18:48:34.909966 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:34.909978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:34.910041 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:34.910188 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:34.910189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:34.910385 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:34.910438 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:34.910440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.911273 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.911275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:34.911859 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:34.911862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.912192 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.912193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:34.912244 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:34.912710 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:34.912715 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:34.914569 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
W20260811 18:48:34.914763 28272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:34.909911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:57303 (local address 127.9.215.62:35437)
0811 18:48:34.910012 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:34.910014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:34.910028 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:48:34.910100 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:34.910101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:34.910112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:34.910169 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:34.910497 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.911195 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.911924 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:34.912118 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:34.912163 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:34.914695 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:57303: BlockingRecv error: recv got EOF from 127.9.215.2:57303 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:48:35 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:35 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:35.034933 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:35.029897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:54085)
0811 18:48:35.030044 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:35.030055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:35.030123 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:35.030254 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:35.030254 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:35.030437 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:35.030488 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:35.030490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.031231 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.031233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:35.031855 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:35.031858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.032252 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.032254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:35.032312 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:35.032819 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:35.032824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:35.034836 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
W20260811 18:48:35.035007 28272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:35.029975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:54085 (local address 127.9.215.62:35437)
0811 18:48:35.030041 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:35.030043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:35.030118 (+ 75us) server_negotiation.cc:407] Connection header received
0811 18:48:35.030169 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:35.030170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:35.030182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:35.030235 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:35.030591 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.031130 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.031945 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.032173 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.032229 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:35.034950 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:54085: BlockingRecv error: recv got EOF from 127.9.215.3:54085 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 18:48:35 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:35 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:35.799202 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:35.794267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:35911)
0811 18:48:35.794410 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:35.794420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:35.794476 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:35.794749 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:35.794750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:35.794931 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:35.794980 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:35.794981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.795742 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.795743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:35.796321 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:35.796323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.796731 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.796732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:35.796791 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:35.797263 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:35.797268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:35.799119 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
W20260811 18:48:35.799332 28276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:35.794314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:35911 (local address 127.9.215.62:35437)
0811 18:48:35.794581 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:35.794583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:35.794594 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:35.794622 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:35.794623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:35.794634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:35.794707 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:35.795082 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.795648 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.796418 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.796634 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.796698 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:35.799265 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:35911: BlockingRecv error: recv got EOF from 127.9.215.1:35911 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":83,"threads_started":1}
Aug 11 18:48:35 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:35 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:35.920243 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:35.915278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:60725)
0811 18:48:35.915438 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:35.915450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:35.915547 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:35.915722 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:35.915723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:35.915896 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:35.915946 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:35.915948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.916595 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.916596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:35.917133 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:35.917135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.917478 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.917479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:35.917537 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:35.917994 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:35.917999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:35.920161 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
W20260811 18:48:35.920342 28276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:35.915447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:60725 (local address 127.9.215.62:35437)
0811 18:48:35.915528 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:35.915530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:35.915539 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:35.915644 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:35.915645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:35.915656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:35.915704 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:35.916022 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.916521 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.917216 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:35.917417 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:35.917473 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:35.920286 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:60725: BlockingRecv error: recv got EOF from 127.9.215.2:60725 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 18:48:36 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:36 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:36.040632 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:36.035611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:34497)
0811 18:48:36.035786 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:36.035798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:36.035876 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:36.036043 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:36.036045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:36.036236 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:36.036297 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:36.036299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.037016 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.037017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:36.037613 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:36.037616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.038002 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.038004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:36.038060 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:36.038581 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:36.038586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:36.040545 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
W20260811 18:48:36.040825 28276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:36.035740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:34497 (local address 127.9.215.62:35437)
0811 18:48:36.035791 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:36.035792 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:36.035871 (+ 79us) server_negotiation.cc:407] Connection header received
0811 18:48:36.035934 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:36.035935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:36.035948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:36.035988 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:36.036410 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.036929 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.037710 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.037911 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.037968 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:36.040680 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:34497: BlockingRecv error: recv got EOF from 127.9.215.3:34497 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 18:48:36 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:36 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:36.804795 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:36.799792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:57837)
0811 18:48:36.799969 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:36.799980 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:36.800043 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:36.800225 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:36.800226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:36.800403 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:36.800465 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:36.800466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.801204 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.801205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:36.801988 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:36.801993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.802328 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.802328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:36.802367 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:36.802841 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:36.802846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:36.804729 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
W20260811 18:48:36.804895 28283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:36.799923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:57837 (local address 127.9.215.62:35437)
0811 18:48:36.800039 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:36.800041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:36.800052 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:36.800103 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:36.800104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:36.800115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:36.800180 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:36.800551 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.801133 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.802056 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.802268 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.802325 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:36.804825 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:57837: BlockingRecv error: recv got EOF from 127.9.215.1:57837 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
Aug 11 18:48:36 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:36 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:36.925432 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:36.920832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:56381)
0811 18:48:36.921007 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:36.921019 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:36.921093 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:36.921212 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:36.921212 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:36.921381 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:36.921433 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:36.921435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.922061 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.922062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:36.922828 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:36.922831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.923159 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.923160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:36.923196 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:36.923650 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:36.923654 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:36.925370 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
W20260811 18:48:36.925539 28283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:36.920940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:56381 (local address 127.9.215.62:35437)
0811 18:48:36.920994 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:36.920996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:36.921088 (+ 92us) server_negotiation.cc:407] Connection header received
0811 18:48:36.921146 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:36.921147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:36.921156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:36.921205 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:36.921484 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.921991 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.922902 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:36.923102 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:36.923155 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:36.925475 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:56381: BlockingRecv error: recv got EOF from 127.9.215.2:56381 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 11 18:48:37 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:37 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:37.046219 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:37.041252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:40489)
0811 18:48:37.041423 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:37.041435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:37.041501 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:37.041694 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:37.041695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:37.041858 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:37.041912 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:37.041914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.042601 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.042602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:37.043183 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:37.043185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.043596 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.043597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:37.043672 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:37.044218 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:37.044224 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:37.046157 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
W20260811 18:48:37.046355 28283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:37.041370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:40489 (local address 127.9.215.62:35437)
0811 18:48:37.041465 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:37.041467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:37.041484 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:48:37.041560 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:37.041561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:37.041574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:37.041629 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:37.041985 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.042519 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.043292 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.043518 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.043578 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:37.046282 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:40489: BlockingRecv error: recv got EOF from 127.9.215.3:40489 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:48:37 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:37 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:37.810436 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:37.805334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:52221)
0811 18:48:37.805588 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:37.805600 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:37.805671 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:37.805872 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:37.805873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:37.806143 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:37.806199 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:37.806203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.806980 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.806981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:37.807633 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:37.807635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.808070 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.808071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:37.808132 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:37.808627 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:37.808631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:37.810369 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":104,"threads_started":1}
W20260811 18:48:37.810585 28294 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:37.805460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:52221 (local address 127.9.215.62:35437)
0811 18:48:37.805579 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:37.805580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:37.805669 (+ 89us) server_negotiation.cc:407] Connection header received
0811 18:48:37.805733 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:37.805734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:37.805752 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:37.805831 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:37.806319 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.806877 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.807736 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.807962 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.808027 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:37.810500 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:52221: BlockingRecv error: recv got EOF from 127.9.215.1:52221 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
Aug 11 18:48:37 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:37 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:37.930751 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:37.925958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:58279)
0811 18:48:37.926114 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:37.926125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:37.926188 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:37.926338 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:37.926339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:37.926511 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:37.926560 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:37.926563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.927269 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.927270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:37.927884 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:37.927887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.928248 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.928249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:37.928307 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:37.928930 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:37.928935 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:37.930677 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
W20260811 18:48:37.930853 28294 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:37.926009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:58279 (local address 127.9.215.62:35437)
0811 18:48:37.926123 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:37.926125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:37.926168 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:48:37.926259 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:37.926260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:37.926272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:37.926323 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:37.926672 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.927191 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.927956 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:37.928181 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:37.928239 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:37.930789 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:58279: BlockingRecv error: recv got EOF from 127.9.215.2:58279 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:48:38 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:38 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:38.051781 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:38.046808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:39615)
0811 18:48:38.047021 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:38.047036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:38.047130 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:38.047266 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:38.047267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:38.047451 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:38.047543 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:38.047545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.048217 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.048218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:38.048782 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:38.048784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.049127 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.049128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:38.049190 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:38.049747 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:38.049752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:38.051698 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":115,"threads_started":1}
W20260811 18:48:38.051914 28294 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:38.046958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:39615 (local address 127.9.215.62:35437)
0811 18:48:38.047042 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:38.047044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:38.047109 (+ 65us) server_negotiation.cc:407] Connection header received
0811 18:48:38.047196 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:38.047196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:38.047211 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:38.047260 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:38.047642 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.048138 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.048868 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.049056 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.049111 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:38.051852 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:39615: BlockingRecv error: recv got EOF from 127.9.215.3:39615 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 18:48:38 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:38 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:38.815857 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:38.810979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:42103)
0811 18:48:38.811165 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:38.811176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:38.811248 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:38.811440 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:38.811442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:38.811652 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:38.811706 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:38.811708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.812365 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.812366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:38.812926 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:38.812928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.813285 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.813287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:38.813341 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:38.813864 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:38.813869 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:38.815774 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
W20260811 18:48:38.815953 28308 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:38.811147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:42103 (local address 127.9.215.62:35437)
0811 18:48:38.811288 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:38.811289 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:38.811298 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:38.811322 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:38.811323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:38.811333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:38.811392 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:38.811772 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.812288 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.812995 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.813194 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.813262 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:38.815888 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:42103: BlockingRecv error: recv got EOF from 127.9.215.1:42103 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":50,"threads_started":1}
Aug 11 18:48:38 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:38 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:38.936524 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:38.931327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:52815)
0811 18:48:38.931543 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:38.931558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:38.931652 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:38.931818 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:38.931819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:38.932029 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:38.932082 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:38.932084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.932932 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.932933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:38.933475 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:38.933477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.933867 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.933868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:38.933927 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:38.934425 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:38.934430 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:38.936461 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
W20260811 18:48:38.936678 28308 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:38.931437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:52815 (local address 127.9.215.62:35437)
0811 18:48:38.931556 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:38.931558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:38.931635 (+ 77us) server_negotiation.cc:407] Connection header received
0811 18:48:38.931726 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:38.931728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:38.931742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:38.931806 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:38.932169 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.932839 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.933591 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:38.933792 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:38.933859 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:38.936611 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:52815: BlockingRecv error: recv got EOF from 127.9.215.2:52815 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 18:48:39 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:39 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:39.057608 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:39.052486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:43933)
0811 18:48:39.052676 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:39.052688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:39.052754 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:39.052932 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:39.052933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:39.053114 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:39.053163 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:39.053165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.053882 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.053883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:39.054446 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:39.054448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.054910 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.054912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:39.054967 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:39.055518 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:39.055524 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:39.057526 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
W20260811 18:48:39.057695 28308 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:39.052661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:43933 (local address 127.9.215.62:35437)
0811 18:48:39.052731 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:39.052732 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:39.052747 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:48:39.052810 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:39.052811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:39.052824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:39.052878 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:39.053258 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.053788 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.054567 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.054794 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.054860 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:39.057639 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:43933: BlockingRecv error: recv got EOF from 127.9.215.3:43933 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 18:48:39 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:39 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:39.824124 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:39.816446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:38183)
0811 18:48:39.816603 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:39.816615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:39.816668 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:39.816816 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:39.816817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:39.816993 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:39.817045 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:39.817047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.817794 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.817796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:39.818355 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:39.818357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.818658 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.818659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:39.818695 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:39.822060 (+ 3365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:39.822065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:39.824051 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
W20260811 18:48:39.824234 28318 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:39.816545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:38183 (local address 127.9.215.62:35437)
0811 18:48:39.816671 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:39.816673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:39.816684 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:39.816712 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:39.816713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:39.816724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:39.816804 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:39.817155 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.817687 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.818422 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.818608 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.818672 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:39.824159 (+ 5487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:38183: BlockingRecv error: recv got EOF from 127.9.215.1:38183 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
Aug 11 18:48:39 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:39 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:39.942659 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:39.937185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:58707)
0811 18:48:39.937373 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:39.937388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:39.937467 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:39.937614 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:39.937615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:39.937817 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:39.937877 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:39.937879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.938723 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.938724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:39.939346 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:39.939350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.939821 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.939822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:39.939878 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:39.940422 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:39.940428 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:39.942593 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":116,"threads_started":1}
W20260811 18:48:39.942781 28318 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:39.937350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:58707 (local address 127.9.215.62:35437)
0811 18:48:39.937450 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:39.937452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:39.937464 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:48:39.937526 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:39.937527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:39.937538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:39.937593 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:39.938000 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.938607 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.939495 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:39.939731 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:39.939800 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:39.942709 (+ 2909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:58707: BlockingRecv error: recv got EOF from 127.9.215.2:58707 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 18:48:40 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:40 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:40.063222 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:40.058245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:45477)
0811 18:48:40.058413 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:40.058425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:40.058502 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:40.058644 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:40.058645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:40.058839 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:40.058889 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:40.058891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.059673 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.059674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:40.060256 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:40.060260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.060705 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.060706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:40.060764 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:40.061274 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:40.061278 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:40.063156 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
W20260811 18:48:40.063416 28318 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:40.058320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:45477 (local address 127.9.215.62:35437)
0811 18:48:40.058379 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:40.058381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:40.058490 (+ 109us) server_negotiation.cc:407] Connection header received
0811 18:48:40.058568 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:40.058569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:40.058583 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:40.058632 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:40.058963 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.059572 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.060365 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.060606 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.060679 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:40.063329 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:45477: BlockingRecv error: recv got EOF from 127.9.215.3:45477 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 18:48:40 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:40 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:40.831372 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:40.824674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:54401)
0811 18:48:40.824866 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:40.824879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:40.824968 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:40.825156 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:40.825157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:40.825398 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:40.825468 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:40.825470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.826469 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.826470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:40.827345 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:40.827349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.827870 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.827871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:40.827925 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:40.828610 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:40.828617 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:40.831249 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
W20260811 18:48:40.831624 28323 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:40.824781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:54401 (local address 127.9.215.62:35437)
0811 18:48:40.824934 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:40.824935 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:40.824947 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:48:40.825054 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:40.825055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:40.825076 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:40.825138 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:40.825577 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.826339 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.827517 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.827774 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.827862 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:40.831414 (+ 3552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:54401: BlockingRecv error: recv got EOF from 127.9.215.1:54401 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
Aug 11 18:48:40 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:40 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:40.949618 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:40.943363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:34465)
0811 18:48:40.943586 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:40.943598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:40.943699 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:40.943885 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:40.943886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:40.944176 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:40.944229 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:40.944231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.945131 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.945133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:40.945852 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:40.945855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.946354 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.946355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:40.946443 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:40.947132 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:40.947137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:40.949508 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
W20260811 18:48:40.949743 28323 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:40.943564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:34465 (local address 127.9.215.62:35437)
0811 18:48:40.943740 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:40.943742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:40.943754 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:48:40.943778 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:40.943779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:40.943790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:40.943864 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:40.944343 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.945000 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.945956 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:40.946231 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:40.946320 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:40.949670 (+ 3350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:34465: BlockingRecv error: recv got EOF from 127.9.215.2:34465 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 11 18:48:41 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:41 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:41.069387 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:41.063822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:34855)
0811 18:48:41.064032 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:41.064055 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:41.064128 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:41.064374 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:41.064376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:41.064599 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:41.064677 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:41.064679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.065393 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.065394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:41.065997 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:41.066000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.066393 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.066394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:41.066450 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:41.066914 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:41.066918 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:41.069292 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
W20260811 18:48:41.069501 28323 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:41.063997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:34855 (local address 127.9.215.62:35437)
0811 18:48:41.064130 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:41.064132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:41.064164 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:48:41.064199 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:41.064199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:41.064221 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:41.064301 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:41.064782 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.065307 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.066076 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.066320 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.066384 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:41.069409 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:34855: BlockingRecv error: recv got EOF from 127.9.215.3:34855 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 11 18:48:41 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:41 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:41.836884 28329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:41.831985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:39519)
0811 18:48:41.832157 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:41.832169 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:41.832261 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:41.832430 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:41.832431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:41.832626 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:41.832682 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:41.832684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.833414 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.833416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:41.833999 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:41.834001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.834377 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.834378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:41.834441 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:41.834914 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:41.834919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:41.836800 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":73,"threads_started":1}
W20260811 18:48:41.836957 28330 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:41.832111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:39519 (local address 127.9.215.62:35437)
0811 18:48:41.832227 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:41.832229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:41.832240 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:41.832345 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:41.832346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:41.832356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:41.832406 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:41.832746 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.833269 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.834079 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.834288 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.834340 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:41.836892 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:39519: BlockingRecv error: recv got EOF from 127.9.215.1:39519 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
Aug 11 18:48:41 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:41 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:41.955166 28331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:41.950238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:32913)
0811 18:48:41.950409 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:41.950422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:41.950510 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:41.950657 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:41.950658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:41.950860 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:41.950909 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:41.950910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.951676 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.951678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:41.952225 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:41.952228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.952574 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.952576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:41.952633 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:41.953148 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:41.953153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:41.955066 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
W20260811 18:48:41.955262 28330 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:41.950376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:32913 (local address 127.9.215.62:35437)
0811 18:48:41.950437 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:41.950439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:41.950487 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:48:41.950585 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:41.950586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:41.950597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:41.950639 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:41.951000 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.951585 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.952312 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:41.952510 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:41.952573 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:41.955193 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:32913: BlockingRecv error: recv got EOF from 127.9.215.2:32913 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 18:48:42 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:42 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:42.074733 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:42.069999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:60525)
0811 18:48:42.070200 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:42.070212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:42.070297 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:42.070423 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:42.070424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:42.070615 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:42.070662 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:42.070664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.071332 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.071334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:42.071916 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:42.071919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.072266 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.072267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:42.072328 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:42.072818 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:42.072822 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:42.074648 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":119,"threads_started":1}
W20260811 18:48:42.074822 28330 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:42.070091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:60525 (local address 127.9.215.62:35437)
0811 18:48:42.070157 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:42.070159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:42.070283 (+ 124us) server_negotiation.cc:407] Connection header received
0811 18:48:42.070354 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:42.070355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:42.070370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:42.070418 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:42.070738 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.071258 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.072002 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.072203 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.072259 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:42.074765 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:60525: BlockingRecv error: recv got EOF from 127.9.215.3:60525 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 11 18:48:42 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:42 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:42.842034 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:42.837352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:48923)
0811 18:48:42.837476 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:42.837487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:42.837556 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:42.837731 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:42.837732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:42.837943 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:42.837995 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:42.837997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.838668 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.838668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:42.839229 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:42.839230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.839582 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.839583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:42.839621 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:42.840076 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:42.840081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:42.841951 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
W20260811 18:48:42.842128 28337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:42.837465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:48923 (local address 127.9.215.62:35437)
0811 18:48:42.837596 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:42.837598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:42.837607 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:48:42.837633 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:42.837633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:42.837646 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:42.837716 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:42.838061 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.838581 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.839294 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.839529 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.839582 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:42.842064 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:48923: BlockingRecv error: recv got EOF from 127.9.215.1:48923 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Aug 11 18:48:42 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:42 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:42.960790 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:42.955794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:44345)
0811 18:48:42.955946 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:42.955958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:42.956031 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:42.956201 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:42.956202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:42.956386 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:42.956435 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:42.956437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.957114 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.957115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:42.957751 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:42.957753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.958151 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.958153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:42.958228 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:42.958803 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:42.958809 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:42.960712 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260811 18:48:42.960888 28337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:42.955920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:44345 (local address 127.9.215.62:35437)
0811 18:48:42.956018 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:42.956020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:42.956031 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:42.956084 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:42.956085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:42.956096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:42.956148 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:42.956513 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.957018 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.957834 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:42.958062 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:42.958121 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:42.960822 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:44345: BlockingRecv error: recv got EOF from 127.9.215.2:44345 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 18:48:43 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:43 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:43.080410 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:43.075334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:50903)
0811 18:48:43.075514 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:43.075529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:43.075630 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:43.075782 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:43.075782 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:43.075967 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:43.076014 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:43.076016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.076740 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.076742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:43.077310 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:43.077312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.077661 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.077662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:43.077721 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:43.078224 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:43.078228 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:43.080299 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":67,"threads_started":1}
W20260811 18:48:43.080459 28337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:43.075485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:50903 (local address 127.9.215.62:35437)
0811 18:48:43.075544 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:43.075546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:43.075584 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:48:43.075700 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:43.075701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:43.075715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:43.075766 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:43.076088 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.076661 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.077391 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.077597 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.077654 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:43.080401 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:50903: BlockingRecv error: recv got EOF from 127.9.215.3:50903 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 18:48:43 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:43 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:43.847363 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:43.842711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:34885)
0811 18:48:43.842867 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:43.842878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:43.842928 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:43.843069 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:43.843070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:43.843238 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:43.843288 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:43.843289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.843949 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.843950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:43.844515 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:43.844518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.844948 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.844949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:43.845005 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:43.845471 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:43.845477 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:43.847297 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
W20260811 18:48:43.847582 28346 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:43.842789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:34885 (local address 127.9.215.62:35437)
0811 18:48:43.842908 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:43.842909 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:43.842919 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:43.842976 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:43.842977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:43.842987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:43.843052 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:43.843343 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.843870 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.844633 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.844850 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.844913 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:43.847486 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:34885: BlockingRecv error: recv got EOF from 127.9.215.1:34885 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
Aug 11 18:48:43 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:43 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:43.966208 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:43.961310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:53277)
0811 18:48:43.961639 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:43.961650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:43.961721 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:43.961857 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:43.961858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:43.962042 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:43.962091 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:43.962093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.962755 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.962757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:43.963285 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:43.963287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.963705 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.963706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:43.963764 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:43.964245 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:43.964249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:43.966128 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":265,"threads_started":1}
W20260811 18:48:43.966310 28346 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:43.961464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:53277 (local address 127.9.215.62:35437)
0811 18:48:43.961535 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:43.961538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:43.961720 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:48:43.961781 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:43.961783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:43.961796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:43.961844 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:43.962174 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.962674 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.963369 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:43.963629 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:43.963684 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:43.966252 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:53277: BlockingRecv error: recv got EOF from 127.9.215.2:53277 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 18:48:44 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:44 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:44.085973 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:44.081142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:39751)
0811 18:48:44.081325 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:44.081338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:44.081435 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:44.081605 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:44.081606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:44.081824 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:44.081880 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:44.081882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.082579 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.082580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:44.083130 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:44.083133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.083497 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.083498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:44.083549 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:44.083982 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:44.083987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:44.085888 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
W20260811 18:48:44.086052 28346 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:44.081187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:39751 (local address 127.9.215.62:35437)
0811 18:48:44.081252 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:44.081255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:44.081411 (+ 156us) server_negotiation.cc:407] Connection header received
0811 18:48:44.081508 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:44.081509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:44.081523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:44.081582 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:44.081954 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.082502 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.083209 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.083404 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.083479 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:44.085995 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:39751: BlockingRecv error: recv got EOF from 127.9.215.3:39751 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 18:48:44 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:44 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:44.859071 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:44.848051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:34729)
0811 18:48:44.848238 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:44.848251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:44.848315 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:44.848483 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:44.848484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:44.848662 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:44.848715 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:44.848717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.849519 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.849520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:44.850092 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:44.850094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.850486 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.850487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:44.850525 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:44.857096 (+ 6571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:44.857104 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:44.859002 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
W20260811 18:48:44.859211 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:44.848191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:34729 (local address 127.9.215.62:35437)
0811 18:48:44.848314 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:44.848316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:44.848345 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:48:44.848372 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:44.848373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:44.848383 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:44.848433 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:44.848837 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.849407 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.850184 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.850418 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.850483 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:44.859139 (+ 8656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:34729: BlockingRecv error: recv got EOF from 127.9.215.1:34729 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":52,"threads_started":1}
Aug 11 18:48:44 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:44 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:44.972035 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:44.966810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:43919)
0811 18:48:44.966960 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:44.966976 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:44.967052 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:44.967318 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:44.967320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:44.967532 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:44.967587 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:44.967588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.968320 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.968322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:44.968953 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:44.968956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.969359 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.969360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:44.969423 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:44.969910 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:44.969915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:44.971954 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
W20260811 18:48:44.972210 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:44.966933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:43919 (local address 127.9.215.62:35437)
0811 18:48:44.967022 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:44.967024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:44.967037 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:48:44.967172 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:44.967174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:44.967187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:44.967249 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:44.967690 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.968218 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.969055 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:44.969265 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:44.969328 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:44.972145 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:43919: BlockingRecv error: recv got EOF from 127.9.215.2:43919 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 18:48:45 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:45 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.3@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:45.091430 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:45.086596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.3:51531)
0811 18:48:45.086806 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:45.086816 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:45.086881 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:45.087021 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:45.087021 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:45.087212 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:45.087262 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:45.087263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.088082 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.088083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:45.088619 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:45.088621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.089004 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.089005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:45.089064 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:45.089512 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:45.089516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:45.091364 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
W20260811 18:48:45.091672 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:45.086641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:51531 (local address 127.9.215.62:35437)
0811 18:48:45.086698 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:45.086700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:45.086862 (+ 162us) server_negotiation.cc:407] Connection header received
0811 18:48:45.086936 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:45.086937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:45.086951 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:45.087002 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:45.087337 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.087998 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.088703 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.088938 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.089001 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:45.091578 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.3:51531: BlockingRecv error: recv got EOF from 127.9.215.3:51531 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 18:48:45 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:45 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.1@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:45.864549 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:45.859611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.1:42437)
0811 18:48:45.859793 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:45.859805 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:45.859865 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:45.860031 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:45.860033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:45.860201 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:45.860252 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:45.860253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.860959 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.860959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:45.861508 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:45.861510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.861897 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.861898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:45.861946 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:45.862428 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:45.862433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:45.864473 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
W20260811 18:48:45.864723 28363 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:45.859704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:42437 (local address 127.9.215.62:35437)
0811 18:48:45.859856 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:45.859858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:45.859868 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:45.859911 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:45.859911 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:45.859921 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:45.859981 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:45.860340 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.860898 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.861613 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.861827 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.861883 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:45.864615 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.1:42437: BlockingRecv error: recv got EOF from 127.9.215.1:42437 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
Aug 11 18:48:45 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:48:45 dist-test-slave-pww0 krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.215.2@KRBTEST.COM for oryx/127.9.215.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:48:45.977355 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:45.972670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35437 (local address 127.9.215.2:39849)
0811 18:48:45.972817 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:45.972830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:45.972896 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:45.973034 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:45.973035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:45.973214 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:45.973266 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:45.973269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.973948 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.973949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:45.974488 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:45.974490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.974849 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.974850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:45.974908 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:45.975414 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:45.975419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:45.977289 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.215.62:35437: Server oryx/127.9.215.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
W20260811 18:48:45.977465 28363 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:45.972795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:39849 (local address 127.9.215.62:35437)
0811 18:48:45.972863 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:45.972865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:45.972877 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:48:45.972965 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:45.972966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:45.972978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:45.973026 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:45.973344 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.973864 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.974572 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:45.974781 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:45.974839 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:45.977401 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.215.2:39849: BlockingRecv error: recv got EOF from 127.9.215.2:39849 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:48:46.012805 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 27360
I20260811 18:48:46.067884 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 27674
I20260811 18:48:46.084301 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 27951
I20260811 18:48:46.098114 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 27223
2026-08-11T18:48:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17823 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T18:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:46Z Disabled control of system clock
I20260811 18:48:46.140388 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35421
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:35421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:48:46.229293 28377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:46.229532 28377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:46.229552 28377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:46.231109 28377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:46.231184 28377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:46.231199 28377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:46.231210 28377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:46.231221 28377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:46.232844 28377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:35421
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:46.233176 28377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:46.233485 28377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:46.238624 28391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.238623 28389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.238864 28377 server_base.cc:1034] running on GCE node
W20260811 18:48:46.238658 28388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.239223 28377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:46.239602 28377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:46.240777 28377 hybrid_clock.cc:648] HybridClock initialized: now 1786474126240762 us; error 34 us; skew 500 ppm
I20260811 18:48:46.242138 28377 webserver.cc:460] Webserver started at http://127.9.215.62:33367/ using document root <none> and password file <none>
I20260811 18:48:46.242411 28377 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:46.242467 28377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:46.242601 28377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:46.243878 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "512a9afc16be4e3eb8db252045aee784"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "49046bf6814ae03572b02a9df2e57106"
server_key_iv: "c0e76631ca49f209ee162409fd182989"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.244237 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "512a9afc16be4e3eb8db252045aee784"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "49046bf6814ae03572b02a9df2e57106"
server_key_iv: "c0e76631ca49f209ee162409fd182989"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.245930 28377 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:46.246992 28403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.248023 28377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:46.248158 28377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "512a9afc16be4e3eb8db252045aee784"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "49046bf6814ae03572b02a9df2e57106"
server_key_iv: "c0e76631ca49f209ee162409fd182989"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.248263 28377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:46.265888 28377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:46.266266 28377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:46.266417 28377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:46.270905 28377 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:35421
I20260811 18:48:46.271013 28483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:35421 every 8 connection(s)
I20260811 18:48:46.271334 28377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:46.272125 28485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:46.275156 28485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784: Bootstrap starting.
I20260811 18:48:46.275868 28485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:46.276340 28485 log.cc:824] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:46.276381 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 28377
I20260811 18:48:46.276440 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:46.276614 10076 external_mini_cluster.cc:1428] Setting key 632e41dcab60ca1f589a00b7d8cf5b2c
I20260811 18:48:46.277195 28485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784: No bootstrap required, opened a new log
I20260811 18:48:46.279444 28485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512a9afc16be4e3eb8db252045aee784" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35421 } }
I20260811 18:48:46.279685 28485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:46.279724 28485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512a9afc16be4e3eb8db252045aee784, State: Initialized, Role: FOLLOWER
I20260811 18:48:46.279865 28485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512a9afc16be4e3eb8db252045aee784" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35421 } }
I20260811 18:48:46.279970 28485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:46.280017 28485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:46.280076 28485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:46.280962 28485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512a9afc16be4e3eb8db252045aee784" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35421 } }
I20260811 18:48:46.281090 28485 leader_election.cc:302] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 512a9afc16be4e3eb8db252045aee784; no voters:
I20260811 18:48:46.281265 28485 leader_election.cc:288] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:46.281376 28494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:46.281549 28494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [term 1 LEADER]: Becoming Leader. State: Replica: 512a9afc16be4e3eb8db252045aee784, State: Running, Role: LEADER
I20260811 18:48:46.281603 28485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:46.281653 28494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512a9afc16be4e3eb8db252045aee784" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35421 } }
I20260811 18:48:46.282018 28494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 512a9afc16be4e3eb8db252045aee784. Latest consensus state: current_term: 1 leader_uuid: "512a9afc16be4e3eb8db252045aee784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512a9afc16be4e3eb8db252045aee784" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35421 } } }
I20260811 18:48:46.282112 28494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:46.282033 28499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "512a9afc16be4e3eb8db252045aee784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512a9afc16be4e3eb8db252045aee784" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35421 } } }
I20260811 18:48:46.282232 28499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:46.282513 28506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:46.283280 28506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:46.285218 28506 catalog_manager.cc:1347] Generated new cluster ID: fa59d132c8654e46a15cb1efdd8ff941
I20260811 18:48:46.285300 28506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:46.285393 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.277155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.9.215.62:35421)
0811 18:48:46.277379 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:46.277389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:46.277425 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:48:46.278502 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:46.278513 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:46.278616 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:46.278794 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:46.279028 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.281334 (+ 2306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.282727 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.283408 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.283612 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.284574 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:46.284578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:46.284578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:46.284796 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:46.284832 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:46.284844 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:46.284908 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:46.285012 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 18:48:46.285099 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260811 18:48:46.321710 28506 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:46.322320 28506 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:46.328449 28506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 512a9afc16be4e3eb8db252045aee784: Generated new TSK 0
I20260811 18:48:46.328662 28506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:46.333609 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35421
--builtin_ntp_servers=127.9.215.20:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 18:48:46.436182 28533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:46.436401 28533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:46.436424 28533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:46.437669 28533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:46.437719 28533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:46.437747 28533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:46.439270 28533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:46.439564 28533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:46.439890 28533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:46.445781 28546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.445865 28533 server_base.cc:1034] running on GCE node
W20260811 18:48:46.445794 28549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.445788 28552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.446163 28533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:46.446434 28533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:46.447583 28533 hybrid_clock.cc:648] HybridClock initialized: now 1786474126447550 us; error 48 us; skew 500 ppm
I20260811 18:48:46.448999 28533 webserver.cc:460] Webserver started at http://127.9.215.1:39913/ using document root <none> and password file <none>
I20260811 18:48:46.449257 28533 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:46.449308 28533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:46.449455 28533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:46.450618 28533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "d371c705040344b5b0bb72f5cee6c48e"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "24e20184270d5d2d897910378b7fc419"
server_key_iv: "173a5d1dff72263616df29811c636d81"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.450966 28533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "d371c705040344b5b0bb72f5cee6c48e"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "24e20184270d5d2d897910378b7fc419"
server_key_iv: "173a5d1dff72263616df29811c636d81"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.452428 28533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:46.453289 28562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.453470 28533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:46.453554 28533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "d371c705040344b5b0bb72f5cee6c48e"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "24e20184270d5d2d897910378b7fc419"
server_key_iv: "173a5d1dff72263616df29811c636d81"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.453649 28533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:46.473946 28533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:46.474323 28533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:46.474472 28533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:46.474735 28533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:46.475082 28533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:46.475119 28533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.475149 28533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:46.475167 28533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.481282 28533 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:36651
I20260811 18:48:46.481431 28788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:36651 every 8 connection(s)
I20260811 18:48:46.481748 28533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:46.487113 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.482485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:46225 (local address 127.9.215.62:35421)
0811 18:48:46.482551 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:46.482553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:46.482826 (+ 273us) server_negotiation.cc:407] Connection header received
0811 18:48:46.483346 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:46.483347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:46.483363 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:46.483421 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:46.484143 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.484652 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.485706 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.485913 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.485985 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.486881 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:46.486882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:46.486883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:46.486887 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:46.486904 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:46.486908 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:46.486953 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:46.487013 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 18:48:46.487038 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:48:46.487360 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.482445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35421 (local address 127.9.215.1:46225)
0811 18:48:46.482652 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:46.482718 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:46.483255 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:46.483496 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:46.483500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:46.483523 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:46.484056 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:46.484061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.484769 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.484770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:46.485597 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:46.485600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.485996 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.485997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:46.486120 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.486727 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:46.486727 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:46.486760 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:46.486787 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:46.486954 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:46.486957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:46.486960 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:46.487012 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:48:46.487135 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260811 18:48:46.487892 28789 heartbeater.cc:347] Connected to a master server at 127.9.215.62:35421
I20260811 18:48:46.487986 28789 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:46.488157 28789 heartbeater.cc:511] Master 127.9.215.62:35421 requested a full tablet report, sending...
I20260811 18:48:46.488631 28429 ts_manager.cc:194] Registered new tserver with Master: d371c705040344b5b0bb72f5cee6c48e (127.9.215.1:36651)
I20260811 18:48:46.489269 28429 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.1:46225
I20260811 18:48:46.491173 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 28533
I20260811 18:48:46.491259 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:46.491398 10076 external_mini_cluster.cc:1428] Setting key 0ec82bae0d277707a3533a1da155ee33
I20260811 18:48:46.492333 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35421
--builtin_ntp_servers=127.9.215.20:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 18:48:46.576896 28795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:46.577148 28795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:46.577181 28795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:46.578430 28795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:46.578481 28795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:46.578517 28795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:46.580003 28795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:46.580286 28795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:46.580613 28795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:46.585906 28804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.585927 28810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.586100 28795 server_base.cc:1034] running on GCE node
W20260811 18:48:46.585932 28806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.586388 28795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:46.586632 28795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:46.587780 28795 hybrid_clock.cc:648] HybridClock initialized: now 1786474126587761 us; error 35 us; skew 500 ppm
I20260811 18:48:46.589080 28795 webserver.cc:460] Webserver started at http://127.9.215.2:35469/ using document root <none> and password file <none>
I20260811 18:48:46.589316 28795 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:46.589365 28795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:46.589478 28795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:46.590622 28795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "9039bd40a4864912aca4d9a7b39f72e1"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "411270ca59fade520301c9ab95dffb83"
server_key_iv: "c7b89275fc096ad1a43324f4a25210aa"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.591001 28795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "9039bd40a4864912aca4d9a7b39f72e1"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "411270ca59fade520301c9ab95dffb83"
server_key_iv: "c7b89275fc096ad1a43324f4a25210aa"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.592440 28795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:46.593225 28818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.593400 28795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:48:46.593497 28795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "9039bd40a4864912aca4d9a7b39f72e1"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "411270ca59fade520301c9ab95dffb83"
server_key_iv: "c7b89275fc096ad1a43324f4a25210aa"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.593561 28795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:46.608343 28795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:46.608685 28795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:46.608820 28795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:46.609045 28795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:46.609393 28795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:46.609429 28795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.609459 28795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:46.609483 28795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.615448 28795 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:35609
I20260811 18:48:46.615532 29033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:35609 every 8 connection(s)
I20260811 18:48:46.615882 28795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:46.618016 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 28795
I20260811 18:48:46.618109 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:46.618297 10076 external_mini_cluster.cc:1428] Setting key 6b385ae073d0f478292be381bff5d1a9
I20260811 18:48:46.619268 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35421
--builtin_ntp_servers=127.9.215.20:33467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:48:46.621635 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.616617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:48741 (local address 127.9.215.62:35421)
0811 18:48:46.616679 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:46.616681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:46.617038 (+ 357us) server_negotiation.cc:407] Connection header received
0811 18:48:46.617538 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:46.617539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:46.617550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:46.617603 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:46.618316 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.618925 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.620063 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.620327 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.620401 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.621359 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:46.621360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:46.621361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:46.621366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:46.621383 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:46.621387 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:46.621435 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:46.621521 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 18:48:46.621552 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:48:46.621899 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.616609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35421 (local address 127.9.215.2:48741)
0811 18:48:46.616913 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:46.616950 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:46.617442 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:46.617661 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:46.617665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:46.617681 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:46.618223 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:46.618230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.619013 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.619015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:46.619969 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:46.619973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.620443 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.620445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:46.620580 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.621195 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:46.621196 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:46.621227 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:46.621252 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:46.621442 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:46.621448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:46.621451 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:46.621518 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:48:46.621656 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":199,"threads_started":1}
I20260811 18:48:46.622426 29039 heartbeater.cc:347] Connected to a master server at 127.9.215.62:35421
I20260811 18:48:46.622522 29039 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:46.622722 29039 heartbeater.cc:511] Master 127.9.215.62:35421 requested a full tablet report, sending...
I20260811 18:48:46.623119 28429 ts_manager.cc:194] Registered new tserver with Master: 9039bd40a4864912aca4d9a7b39f72e1 (127.9.215.2:35609)
I20260811 18:48:46.623615 28429 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.2:48741
W20260811 18:48:46.706324 29044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:46.706560 29044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:46.706591 29044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:46.707857 29044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:46.707907 29044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:46.707943 29044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:46.709429 29044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:35421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:46.709710 29044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:46.710029 29044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:46.715664 29044 server_base.cc:1034] running on GCE node
W20260811 18:48:46.715580 29059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.715607 29054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.715806 29055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.716068 29044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:46.716354 29044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:46.717521 29044 hybrid_clock.cc:648] HybridClock initialized: now 1786474126717501 us; error 34 us; skew 500 ppm
I20260811 18:48:46.718953 29044 webserver.cc:460] Webserver started at http://127.9.215.3:32935/ using document root <none> and password file <none>
I20260811 18:48:46.719228 29044 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:46.719277 29044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:46.719408 29044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:46.720572 29044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "b8bb8f5839634688b78f926a6316d7da"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "f532d09b4f7dc7e8d822c7d0beac29c3"
server_key_iv: "6c3355ba9ae041d4a92564f1ad50d638"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.720912 29044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "b8bb8f5839634688b78f926a6316d7da"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "f532d09b4f7dc7e8d822c7d0beac29c3"
server_key_iv: "6c3355ba9ae041d4a92564f1ad50d638"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.722292 29044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:48:46.723191 29069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.723438 29044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:46.723573 29044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "b8bb8f5839634688b78f926a6316d7da"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "f532d09b4f7dc7e8d822c7d0beac29c3"
server_key_iv: "6c3355ba9ae041d4a92564f1ad50d638"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.723652 29044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:46.742157 29044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:46.742516 29044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:46.742662 29044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:46.742913 29044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:46.743284 29044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:46.743322 29044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.743355 29044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:46.743378 29044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.749841 29044 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:33661
I20260811 18:48:46.749890 29315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:33661 every 8 connection(s)
I20260811 18:48:46.750245 29044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:46.756142 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 29044
I20260811 18:48:46.756258 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:46.756281 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.751105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:55145 (local address 127.9.215.62:35421)
0811 18:48:46.751163 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:46.751165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:46.751436 (+ 271us) server_negotiation.cc:407] Connection header received
0811 18:48:46.752112 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:46.752113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:46.752125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:46.752213 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:46.752953 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.753467 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.754535 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.754786 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.754852 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.755924 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:46.755926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:46.755926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:46.755932 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:46.755949 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:46.755953 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:46.756019 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:46.756160 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 18:48:46.756193 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:48:46.756460 10076 external_mini_cluster.cc:1428] Setting key df18fab16557edc2f208edfa948603e9
I20260811 18:48:46.756565 29320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:46.751059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:35421 (local address 127.9.215.3:55145)
0811 18:48:46.751298 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:46.751336 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:46.751978 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:46.752301 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:46.752306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:46.752315 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:46.752853 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:46.752858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.753568 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.753569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:46.754429 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:46.754431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:46.754878 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:46.754878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:46.755011 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:46.755752 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:46.755752 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:46.755787 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:46.755814 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:46.756050 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:46.756059 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:46.756062 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:46.756158 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:48:46.756320 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260811 18:48:46.757153 29316 heartbeater.cc:347] Connected to a master server at 127.9.215.62:35421
I20260811 18:48:46.757265 29316 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:46.757488 29316 heartbeater.cc:511] Master 127.9.215.62:35421 requested a full tablet report, sending...
I20260811 18:48:46.757841 28429 ts_manager.cc:194] Registered new tserver with Master: b8bb8f5839634688b78f926a6316d7da (127.9.215.3:33661)
I20260811 18:48:46.758213 28429 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.3:55145
I20260811 18:48:46.768430 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:48:46.770460 28490 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:46.769876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.9.215.62:35421)
0811 18:48:46.769921 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:46.769924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:46.770052 (+ 128us) server_negotiation.cc:407] Connection header received
0811 18:48:46.770122 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:46.770123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:46.770133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:46.770176 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:46.770370 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41980: BlockingRecv error: recv got EOF from 127.0.0.1:41980 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:48:46.770509 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 28533
I20260811 18:48:46.821455 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 28795
I20260811 18:48:46.832037 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 29044
I20260811 18:48:46.849318 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 28377
2026-08-11T18:48:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (750 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T18:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:46Z Disabled control of system clock
I20260811 18:48:46.889030 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:38639
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:38639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:48:46.976490 29333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:46.976750 29333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:46.976776 29333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:46.978030 29333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:46.978077 29333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:46.978091 29333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:46.978101 29333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:46.978112 29333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:46.979727 29333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:38639
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:38639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:46.980012 29333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:46.980322 29333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:46.985847 29351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.985859 29345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:46.985847 29347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:46.986131 29333 server_base.cc:1034] running on GCE node
I20260811 18:48:46.986398 29333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:46.986723 29333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:46.987883 29333 hybrid_clock.cc:648] HybridClock initialized: now 1786474126987879 us; error 44 us; skew 500 ppm
I20260811 18:48:46.989317 29333 webserver.cc:460] Webserver started at http://127.9.215.62:34473/ using document root <none> and password file <none>
I20260811 18:48:46.989573 29333 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:46.989627 29333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:46.989770 29333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:46.990907 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "5a92fa04a9304fca88ca926ba2ad8545"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "160bc7851b6e40b72d2019d91735f5e0"
server_key_iv: "91c763aadac81c8373473facf8402e2d"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.991426 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "5a92fa04a9304fca88ca926ba2ad8545"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "160bc7851b6e40b72d2019d91735f5e0"
server_key_iv: "91c763aadac81c8373473facf8402e2d"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.993106 29333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:46.993980 29361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:46.994148 29333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:46.994251 29333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "5a92fa04a9304fca88ca926ba2ad8545"
format_stamp: "Formatted at 2026-08-11 18:48:46 on dist-test-slave-pww0"
server_key: "160bc7851b6e40b72d2019d91735f5e0"
server_key_iv: "91c763aadac81c8373473facf8402e2d"
server_key_version: "encryptionkey@0"
I20260811 18:48:46.994331 29333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:46.998001 29333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:46.998298 29333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:46.998425 29333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:47.002658 29333 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:38639
I20260811 18:48:47.002751 29462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:38639 every 8 connection(s)
I20260811 18:48:47.003095 29333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:47.003875 29465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:47.004554 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 29333
I20260811 18:48:47.004647 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:47.004814 10076 external_mini_cluster.cc:1428] Setting key 3c21edaf31446a9d070a33f33d1fdfca
I20260811 18:48:47.007067 29465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545: Bootstrap starting.
I20260811 18:48:47.007840 29465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:47.008221 29465 log.cc:824] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:47.009148 29465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545: No bootstrap required, opened a new log
I20260811 18:48:47.009714 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.005378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.9.215.62:38639)
0811 18:48:47.005586 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:47.005597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:47.007529 (+ 1932us) server_negotiation.cc:407] Connection header received
0811 18:48:47.008169 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:47.008181 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:47.008245 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:47.008419 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:47.009224 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:47.009229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:47.009229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:47.009245 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:47.009305 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:47.009323 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:47.009412 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:47.009445 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 18:48:47.009508 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260811 18:48:47.011248 29465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a92fa04a9304fca88ca926ba2ad8545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38639 } }
I20260811 18:48:47.011416 29465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:47.011440 29465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a92fa04a9304fca88ca926ba2ad8545, State: Initialized, Role: FOLLOWER
I20260811 18:48:47.011598 29465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a92fa04a9304fca88ca926ba2ad8545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38639 } }
I20260811 18:48:47.011684 29465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:47.011719 29465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:47.011778 29465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:47.012418 29465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a92fa04a9304fca88ca926ba2ad8545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38639 } }
I20260811 18:48:47.012521 29465 leader_election.cc:302] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a92fa04a9304fca88ca926ba2ad8545; no voters:
I20260811 18:48:47.012727 29465 leader_election.cc:288] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:47.012845 29472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:47.013049 29465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:47.013100 29472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [term 1 LEADER]: Becoming Leader. State: Replica: 5a92fa04a9304fca88ca926ba2ad8545, State: Running, Role: LEADER
I20260811 18:48:47.013242 29472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a92fa04a9304fca88ca926ba2ad8545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38639 } }
I20260811 18:48:47.013645 29474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a92fa04a9304fca88ca926ba2ad8545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a92fa04a9304fca88ca926ba2ad8545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38639 } } }
I20260811 18:48:47.013743 29474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:47.013899 29474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a92fa04a9304fca88ca926ba2ad8545. Latest consensus state: current_term: 1 leader_uuid: "5a92fa04a9304fca88ca926ba2ad8545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a92fa04a9304fca88ca926ba2ad8545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 38639 } } }
I20260811 18:48:47.013958 29474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:47.014071 29491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:47.014676 29491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:47.016376 29491 catalog_manager.cc:1347] Generated new cluster ID: 88a9c8852e69416d85a3fdd063be8737
I20260811 18:48:47.016445 29491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:47.025406 29491 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:47.025635 29491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:47.027817 29491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a92fa04a9304fca88ca926ba2ad8545: Generated new TSK 0
I20260811 18:48:47.028030 29491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:48:47.031667 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38639
--builtin_ntp_servers=127.9.215.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:48:47.141435 29511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:47.141685 29511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:47.141723 29511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:47.143054 29511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:47.143108 29511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:47.143148 29511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:47.144755 29511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:47.145053 29511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:47.145372 29511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:47.150797 29520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:47.150887 29521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:47.150858 29526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:47.151541 29511 server_base.cc:1034] running on GCE node
I20260811 18:48:47.151834 29511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:47.152184 29511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:47.153384 29511 hybrid_clock.cc:648] HybridClock initialized: now 1786474127153361 us; error 35 us; skew 500 ppm
I20260811 18:48:47.155078 29511 webserver.cc:460] Webserver started at http://127.9.215.1:37631/ using document root <none> and password file <none>
I20260811 18:48:47.155388 29511 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:47.155490 29511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:47.155654 29511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:47.157200 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "283afad8dedb409cb2a8dd862f214711"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "7957b74b53fcf47f22e64bfc98cf71e3"
server_key_iv: "46efecc888feedda58e12ae94fbe2d44"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.157672 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "283afad8dedb409cb2a8dd862f214711"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "7957b74b53fcf47f22e64bfc98cf71e3"
server_key_iv: "46efecc888feedda58e12ae94fbe2d44"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.159525 29511 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 18:48:47.160547 29540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.160753 29511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:47.160858 29511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "283afad8dedb409cb2a8dd862f214711"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "7957b74b53fcf47f22e64bfc98cf71e3"
server_key_iv: "46efecc888feedda58e12ae94fbe2d44"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.160946 29511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:47.166046 29511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:47.166409 29511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:47.166564 29511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:47.166823 29511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:47.167186 29511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:47.167227 29511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.167255 29511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:47.167277 29511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.175051 29511 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:38287
I20260811 18:48:47.175120 29757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:38287 every 8 connection(s)
I20260811 18:48:47.175590 29511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:47.179040 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.176526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:53105 (local address 127.9.215.62:38639)
0811 18:48:47.176588 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:47.176590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:47.176869 (+ 279us) server_negotiation.cc:407] Connection header received
0811 18:48:47.177423 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:47.177425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:47.177433 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:47.177489 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:47.178725 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:47.178727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:47.178727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:47.178732 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:47.178751 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:47.178756 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:47.178808 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:47.178939 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 18:48:47.178957 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:48:47.179055 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 29511
I20260811 18:48:47.179144 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:48:47.179323 10076 external_mini_cluster.cc:1428] Setting key 537d9d6179d6de5508cc61d6b2e55bc9
I20260811 18:48:47.179756 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.176502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:38639 (local address 127.9.215.1:53105)
0811 18:48:47.176734 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:47.176776 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:47.177320 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:47.177570 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:47.177574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:47.177583 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:47.178554 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:47.178555 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:47.178591 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:47.178623 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:47.178814 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:47.178822 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:47.178844 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:48:47.179427 (+ 583us) client_negotiation.cc:240] Negotiation successful
0811 18:48:47.179538 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260811 18:48:47.180411 29758 heartbeater.cc:347] Connected to a master server at 127.9.215.62:38639
I20260811 18:48:47.180446 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38639
--builtin_ntp_servers=127.9.215.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:48:47.180524 29758 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:47.180720 29758 heartbeater.cc:511] Master 127.9.215.62:38639 requested a full tablet report, sending...
I20260811 18:48:47.181274 29401 ts_manager.cc:194] Registered new tserver with Master: 283afad8dedb409cb2a8dd862f214711 (127.9.215.1:38287)
W20260811 18:48:47.268524 29762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:47.268755 29762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:47.268779 29762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:47.270310 29762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:47.270383 29762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:47.270414 29762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:47.272073 29762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:47.272358 29762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:47.272675 29762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:47.278077 29771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:47.278088 29776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:47.278126 29762 server_base.cc:1034] running on GCE node
W20260811 18:48:47.278077 29770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:47.278563 29762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:47.278828 29762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:47.279999 29762 hybrid_clock.cc:648] HybridClock initialized: now 1786474127279984 us; error 37 us; skew 500 ppm
I20260811 18:48:47.281380 29762 webserver.cc:460] Webserver started at http://127.9.215.2:45803/ using document root <none> and password file <none>
I20260811 18:48:47.281641 29762 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:47.281694 29762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:47.281834 29762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:47.282958 29762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "d52ed87a03ef450c92164c3e68428514"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "c6c48eeef98b1f66f196d21d651a13bc"
server_key_iv: "f9d7ae898ba99bf88e0ad0f3913c3698"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.283294 29762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "d52ed87a03ef450c92164c3e68428514"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "c6c48eeef98b1f66f196d21d651a13bc"
server_key_iv: "f9d7ae898ba99bf88e0ad0f3913c3698"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.284621 29762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:47.285331 29788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.285470 29762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:47.285562 29762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "d52ed87a03ef450c92164c3e68428514"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "c6c48eeef98b1f66f196d21d651a13bc"
server_key_iv: "f9d7ae898ba99bf88e0ad0f3913c3698"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.285626 29762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:47.289098 29762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:47.289383 29762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:47.289505 29762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:47.289698 29762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:47.290004 29762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:47.290036 29762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.290066 29762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:47.290089 29762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.296207 29762 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:35071
I20260811 18:48:47.296285 30025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:35071 every 8 connection(s)
I20260811 18:48:47.296674 29762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:48:47.296964 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 29762
I20260811 18:48:47.297070 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:47.297269 10076 external_mini_cluster.cc:1428] Setting key eceea4c4d3a1354cdbbcf8374f303996
I20260811 18:48:47.298339 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38639
--builtin_ntp_servers=127.9.215.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:48:47.300577 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.297786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:43111 (local address 127.9.215.62:38639)
0811 18:48:47.297833 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:47.297835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:47.297843 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:47.298528 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:47.298530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:47.298537 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:47.298564 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:47.300284 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:47.300286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:47.300286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:47.300293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:47.300315 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:47.300321 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:47.300396 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:47.300480 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:48:47.300497 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:48:47.300796 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.297514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:38639 (local address 127.9.215.2:43111)
0811 18:48:47.297719 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:47.297761 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:47.298388 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:47.298759 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:47.298763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:47.298774 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:47.300056 (+ 1282us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:47.300057 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:47.300103 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:47.300147 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:47.300404 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:47.300409 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:47.300413 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:47.300480 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:48:47.300543 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260811 18:48:47.301318 30027 heartbeater.cc:347] Connected to a master server at 127.9.215.62:38639
I20260811 18:48:47.301420 30027 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:47.301585 30027 heartbeater.cc:511] Master 127.9.215.62:38639 requested a full tablet report, sending...
I20260811 18:48:47.301998 29401 ts_manager.cc:194] Registered new tserver with Master: d52ed87a03ef450c92164c3e68428514 (127.9.215.2:35071)
W20260811 18:48:47.386036 30030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:47.386278 30030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:47.386314 30030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:47.387630 30030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:47.387681 30030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:47.387719 30030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:47.389302 30030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:38639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:47.389731 30030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:47.390093 30030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:47.395418 30042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:47.395531 30030 server_base.cc:1034] running on GCE node
W20260811 18:48:47.395416 30046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:47.395429 30044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:47.395870 30030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:47.396163 30030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:47.397320 30030 hybrid_clock.cc:648] HybridClock initialized: now 1786474127397303 us; error 31 us; skew 500 ppm
I20260811 18:48:47.398571 30030 webserver.cc:460] Webserver started at http://127.9.215.3:45049/ using document root <none> and password file <none>
I20260811 18:48:47.398808 30030 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:47.398859 30030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:47.398994 30030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:47.400137 30030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "842d75113b9f4dfe87424970b41b63f3"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "8af1c1dfa628be8f9d0bb650def1c3ff"
server_key_iv: "1e6aa45a8c513a1b9c74fcd849a9f06f"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.400480 30030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "842d75113b9f4dfe87424970b41b63f3"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "8af1c1dfa628be8f9d0bb650def1c3ff"
server_key_iv: "1e6aa45a8c513a1b9c74fcd849a9f06f"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.401765 30030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:48:47.402513 30065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.402683 30030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:48:47.402776 30030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "842d75113b9f4dfe87424970b41b63f3"
format_stamp: "Formatted at 2026-08-11 18:48:47 on dist-test-slave-pww0"
server_key: "8af1c1dfa628be8f9d0bb650def1c3ff"
server_key_iv: "1e6aa45a8c513a1b9c74fcd849a9f06f"
server_key_version: "encryptionkey@0"
I20260811 18:48:47.402839 30030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:47.406303 30030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:47.406576 30030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:47.406705 30030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:47.406901 30030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:47.407208 30030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:47.407243 30030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.407269 30030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:47.407294 30030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:47.414171 30030 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:36781
I20260811 18:48:47.414263 30288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:36781 every 8 connection(s)
I20260811 18:48:47.414634 30030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:48:47.415305 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 30030
I20260811 18:48:47.415375 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:47.415593 10076 external_mini_cluster.cc:1428] Setting key a0dbebf58c0294a5b7219c7af4dbe9d5
I20260811 18:48:47.418174 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.415806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:43201 (local address 127.9.215.62:38639)
0811 18:48:47.415879 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:47.415881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:47.416215 (+ 334us) server_negotiation.cc:407] Connection header received
0811 18:48:47.416732 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:47.416733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:47.416739 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:47.416802 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:47.417956 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:47.417957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:47.417958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:48:47.417962 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:47.417980 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:48:47.417984 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:47.418045 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:47.418097 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 18:48:47.418109 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:48:47.418376 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:47.415813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:38639 (local address 127.9.215.3:43201)
0811 18:48:47.416088 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:47.416122 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:47.416642 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:47.416865 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:47.416869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:47.416878 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:47.417758 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:48:47.417759 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:47.417792 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:48:47.417825 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:47.418046 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:47.418049 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:47.418051 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:48:47.418103 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:48:47.418161 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260811 18:48:47.418829 30289 heartbeater.cc:347] Connected to a master server at 127.9.215.62:38639
I20260811 18:48:47.418928 30289 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:47.419059 30289 heartbeater.cc:511] Master 127.9.215.62:38639 requested a full tablet report, sending...
I20260811 18:48:47.419402 29401 ts_manager.cc:194] Registered new tserver with Master: 842d75113b9f4dfe87424970b41b63f3 (127.9.215.3:36781)
I20260811 18:48:47.427914 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:48:47.429905 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 29511
W20260811 18:48:47.429883 29470 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:48:47.429358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.9.215.62:38639)
0811 18:48:47.429415 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:47.429417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:47.429534 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:48:47.429605 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:47.429606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:47.429612 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:47.429661 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:47.429804 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42878: BlockingRecv error: recv got EOF from 127.0.0.1:42878 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:48:47.500125 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 29762
I20260811 18:48:47.515007 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 30030
I20260811 18:48:47.532956 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 29333
2026-08-11T18:48:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (686 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:47 dist-test-slave-pww0 krb5kdc[30315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:47 dist-test-slave-pww0 krb5kdc[30315](info): set up 2 sockets
Aug 11 18:48:47 dist-test-slave-pww0 krb5kdc[30315](info): commencing operation
krb5kdc: starting...
W20260811 18:48:48.796404 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:48 dist-test-slave-pww0 krb5kdc[30315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:48:48.926321 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45867
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:45867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:49.014304 30343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:49.014559 30343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:49.014583 30343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:49.015942 30343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:48:49.016012 30343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:49.016026 30343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:49.016037 30343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:48:49.016048 30343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:48:49.017863 30343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:45867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:49.018187 30343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:49.018450 30343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:49.023924 30343 server_base.cc:1034] running on GCE node
W20260811 18:48:49.023962 30351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.023901 30355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.023895 30357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:49.024463 30343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:49.024817 30343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:49.025995 30343 hybrid_clock.cc:648] HybridClock initialized: now 1786474129025977 us; error 34 us; skew 500 ppm
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:49.029001 30343 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:48:49.029490 30343 webserver.cc:460] Webserver started at http://127.9.215.62:44209/ using document root <none> and password file <none>
I20260811 18:48:49.029697 30343 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:49.029740 30343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:49.029841 30343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:49.030946 30343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "8053058249b9466ba8a651fd7d1e8743"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "de2c261e04629482fb1bc239641797e9"
server_key_iv: "415374eca6ed9f36e30bf7f6cf7500c6"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.031281 30343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "8053058249b9466ba8a651fd7d1e8743"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "de2c261e04629482fb1bc239641797e9"
server_key_iv: "415374eca6ed9f36e30bf7f6cf7500c6"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.032786 30343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:48:49.033708 30368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.033972 30343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:49.034091 30343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "8053058249b9466ba8a651fd7d1e8743"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "de2c261e04629482fb1bc239641797e9"
server_key_iv: "415374eca6ed9f36e30bf7f6cf7500c6"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.034178 30343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:49.051775 30343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:49.054157 30343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:49.054313 30343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:49.058647 30343 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:45867
I20260811 18:48:49.058693 30455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:45867 every 8 connection(s)
I20260811 18:48:49.059074 30343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:48:49.060302 30457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:49.062744 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 30343
I20260811 18:48:49.062836 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:48:49.062840 30457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743: Bootstrap starting.
I20260811 18:48:49.063001 10076 external_mini_cluster.cc:1428] Setting key f4060c342e48bea8d131e8134e3dbdc3
I20260811 18:48:49.063588 30457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:49.063946 30457 log.cc:824] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:49.064683 30457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743: No bootstrap required, opened a new log
I20260811 18:48:49.066954 30457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8053058249b9466ba8a651fd7d1e8743" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45867 } }
I20260811 18:48:49.067159 30457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:49.067190 30457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8053058249b9466ba8a651fd7d1e8743, State: Initialized, Role: FOLLOWER
I20260811 18:48:49.067337 30457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8053058249b9466ba8a651fd7d1e8743" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45867 } }
I20260811 18:48:49.067497 30457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:48:49.067548 30457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:48:49.067598 30457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:49.068325 30457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8053058249b9466ba8a651fd7d1e8743" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45867 } }
I20260811 18:48:49.068449 30457 leader_election.cc:302] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8053058249b9466ba8a651fd7d1e8743; no voters:
I20260811 18:48:49.068650 30457 leader_election.cc:288] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:48:49.068699 30468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:49.068924 30468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [term 1 LEADER]: Becoming Leader. State: Replica: 8053058249b9466ba8a651fd7d1e8743, State: Running, Role: LEADER
I20260811 18:48:49.069003 30457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:48:49.069053 30468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8053058249b9466ba8a651fd7d1e8743" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45867 } }
I20260811 18:48:49.070106 30468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8053058249b9466ba8a651fd7d1e8743. Latest consensus state: current_term: 1 leader_uuid: "8053058249b9466ba8a651fd7d1e8743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8053058249b9466ba8a651fd7d1e8743" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45867 } } }
I20260811 18:48:49.070120 30470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8053058249b9466ba8a651fd7d1e8743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8053058249b9466ba8a651fd7d1e8743" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45867 } } }
I20260811 18:48:49.070447 30470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [sys.catalog]: This master's current role is: LEADER
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:49.070744 30482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:48:49.070353 30468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743 [sys.catalog]: This master's current role is: LEADER
I20260811 18:48:49.071425 30482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:48:49.073478 30482 catalog_manager.cc:1347] Generated new cluster ID: 5f6b2d8f82104e9f9dcc10017fcd7acc
I20260811 18:48:49.073572 30482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:48:49.075786 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.063603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.9.215.62:45867)
0811 18:48:49.063818 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.063827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.063856 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:48:49.064379 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.064387 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.064446 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.064576 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:49.065483 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.066379 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.067411 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.068092 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.068187 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.071079 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:49.071084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:49.071084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:49.071099 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:49.073062 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.073479 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.073483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.073483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.073617 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.073813 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.073813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.073814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.074055 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:49.074221 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.075389 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.075592 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260811 18:48:49.092334 30482 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:48:49.092926 30482 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:48:49.105312 30482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743: Generated new TSK 0
I20260811 18:48:49.105535 30482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:48:49.142792 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45867
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:49.232600 30500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:49.232858 30500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:49.232887 30500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:49.234141 30500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:49.234186 30500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:49.234217 30500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:48:49.235853 30500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:49.236090 30500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:49.236297 30500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:49.241870 30522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.241842 30516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.241842 30515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:49.242092 30500 server_base.cc:1034] running on GCE node
I20260811 18:48:49.242349 30500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:49.242599 30500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:49.243762 30500 hybrid_clock.cc:648] HybridClock initialized: now 1786474129243742 us; error 34 us; skew 500 ppm
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:49.246464 30500 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:48:49.246933 30500 webserver.cc:460] Webserver started at http://127.9.215.1:45475/ using document root <none> and password file <none>
I20260811 18:48:49.247139 30500 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:49.247184 30500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:49.247280 30500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:49.248409 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "8a983437c1f14c4f99065c64ad6fcbfa"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "1f0572e53b5f52620a81b635494906d7"
server_key_iv: "31ecc501da3f21866a35ab513958ec67"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.248735 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "8a983437c1f14c4f99065c64ad6fcbfa"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "1f0572e53b5f52620a81b635494906d7"
server_key_iv: "31ecc501da3f21866a35ab513958ec67"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.250133 30500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:48:49.250964 30537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.251137 30500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:49.251226 30500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "8a983437c1f14c4f99065c64ad6fcbfa"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "1f0572e53b5f52620a81b635494906d7"
server_key_iv: "31ecc501da3f21866a35ab513958ec67"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.251291 30500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:49.263916 30500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:49.266027 30500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:49.266160 30500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:49.266381 30500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:49.266659 30500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:49.266687 30500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.266708 30500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:49.266724 30500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.273001 30500 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:34959
I20260811 18:48:49.273058 30765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:34959 every 8 connection(s)
I20260811 18:48:49.273408 30500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:48:49.279726 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 30500
I20260811 18:48:49.279832 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:49.280025 10076 external_mini_cluster.cc:1428] Setting key 352f58cf1175784820ab9c1f63632cfd
I20260811 18:48:49.283696 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.274192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:43977 (local address 127.9.215.62:45867)
0811 18:48:49.274270 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.274272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.274457 (+ 185us) server_negotiation.cc:407] Connection header received
0811 18:48:49.275001 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.275002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.275015 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.275071 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:49.276171 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.276710 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.277730 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.277951 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.278008 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.280441 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:49.280443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:49.280443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:49.280452 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:49.282204 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.282670 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.282672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.282672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.282731 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.282991 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.282992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.282992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.283110 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:49.283168 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.283489 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.283601 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:48:49.283890 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.274141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45867 (local address 127.9.215.1:43977)
0811 18:48:49.274323 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.274369 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.274889 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.275165 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.275170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.275842 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:49.276063 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.276068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.276807 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.276808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.277638 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.277641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.278033 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.278034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.278166 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.278678 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:49.278687 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:49.280340 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:49.282303 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:49.282314 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:49.282316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:49.282568 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:49.282800 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:49.282801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:49.282802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:49.282930 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:49.283169 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:49.283177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:49.283337 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.283436 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.283640 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260811 18:48:49.284415 30766 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45867
I20260811 18:48:49.284513 30766 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:49.284706 30766 heartbeater.cc:511] Master 127.9.215.62:45867 requested a full tablet report, sending...
I20260811 18:48:49.285306 30392 ts_manager.cc:194] Registered new tserver with Master: 8a983437c1f14c4f99065c64ad6fcbfa (127.9.215.1:34959)
I20260811 18:48:49.286252 30392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:43977
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:48:49.308432 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45867
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:49.394731 30780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:49.394974 30780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:49.395001 30780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:49.396466 30780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:49.396549 30780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:49.396581 30780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:48:49.398608 30780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:49.398902 30780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:49.399164 30780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:48:49.404838 30798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.404858 30794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:49.405100 30780 server_base.cc:1034] running on GCE node
W20260811 18:48:49.405174 30790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:49.405406 30780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:49.405679 30780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:49.406836 30780 hybrid_clock.cc:648] HybridClock initialized: now 1786474129406821 us; error 28 us; skew 500 ppm
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:49.409550 30780 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:48:49.410015 30780 webserver.cc:460] Webserver started at http://127.9.215.2:43359/ using document root <none> and password file <none>
I20260811 18:48:49.410248 30780 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:49.410315 30780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:49.410454 30780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:49.411597 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "de59f06735f645108946728c99fe3ba7"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "c4016e61c649fb40c8389809fa1cdeab"
server_key_iv: "fbfd78fe9f4b686182880adc1f393b53"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.411938 30780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "de59f06735f645108946728c99fe3ba7"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "c4016e61c649fb40c8389809fa1cdeab"
server_key_iv: "fbfd78fe9f4b686182880adc1f393b53"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.413532 30780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:49.414397 30814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.414640 30780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:48:49.414745 30780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "de59f06735f645108946728c99fe3ba7"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "c4016e61c649fb40c8389809fa1cdeab"
server_key_iv: "fbfd78fe9f4b686182880adc1f393b53"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.414814 30780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:49.435755 30780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:49.438257 30780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:49.438409 30780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:49.438668 30780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:49.438997 30780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:49.439033 30780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.439062 30780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:49.439086 30780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.445039 30780 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:34141
I20260811 18:48:49.445116 31033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:34141 every 8 connection(s)
I20260811 18:48:49.445451 30780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:49.455010 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.446221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:41075 (local address 127.9.215.62:45867)
0811 18:48:49.446276 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.446278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.446475 (+ 197us) server_negotiation.cc:407] Connection header received
0811 18:48:49.447031 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.447033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.447045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.447100 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:49.448214 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.448732 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.449781 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.449978 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.450036 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.452251 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:49.452252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:49.452252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:49.452260 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:49.453650 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.454123 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.454125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.454125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.454205 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.454445 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.454446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.454446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.454557 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:49.454613 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.454841 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.454916 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:48:49.455174 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 30780
I20260811 18:48:49.455194 31039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.446165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45867 (local address 127.9.215.2:41075)
0811 18:48:49.446355 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.446393 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.446928 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.447207 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.447212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.447854 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:49.448084 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.448100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.448842 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.448843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.449681 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.449684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.450048 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.450048 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.450153 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.450716 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:49.450726 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:49.452141 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:49.453752 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:49.453762 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:49.453764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:49.454009 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:49.454271 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:49.454272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:49.454272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:49.454393 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:49.454605 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:49.454609 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:49.454739 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.454821 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.454957 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260811 18:48:49.455281 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:48:49.455513 10076 external_mini_cluster.cc:1428] Setting key ee2b444bec63d16ae212b223d036f481
I20260811 18:48:49.456043 31036 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45867
I20260811 18:48:49.456176 31036 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:49.456408 31036 heartbeater.cc:511] Master 127.9.215.62:45867 requested a full tablet report, sending...
I20260811 18:48:49.457026 30392 ts_manager.cc:194] Registered new tserver with Master: de59f06735f645108946728c99fe3ba7 (127.9.215.2:34141)
I20260811 18:48:49.457441 30392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:41075
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:48:49.484925 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45867
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:48:49.570669 31045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:48:49.570883 31045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:48:49.570909 31045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:48:49.572212 31045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:48:49.572268 31045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:48:49.572299 31045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:48:49.573904 31045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:48:49.574157 31045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:48:49.574399 31045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:48:49.579705 31045 server_base.cc:1034] running on GCE node
W20260811 18:48:49.579715 31054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.579617 31053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:48:49.579622 31057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:48:49.580031 31045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:48:49.580297 31045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:48:49.581461 31045 hybrid_clock.cc:648] HybridClock initialized: now 1786474129581438 us; error 39 us; skew 500 ppm
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:48:49.584100 31045 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:48:49.584617 31045 webserver.cc:460] Webserver started at http://127.9.215.3:46797/ using document root <none> and password file <none>
I20260811 18:48:49.584857 31045 fs_manager.cc:359] Metadata directory not provided
I20260811 18:48:49.584910 31045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:48:49.585041 31045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:48:49.586136 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "0543a4a16bad4cf0ae0a63ba1278348f"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "5103716820a4adab0e55cfd59c7535c8"
server_key_iv: "7f9a18b05f9c00f6d8eddc8911cfea4b"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.586494 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "0543a4a16bad4cf0ae0a63ba1278348f"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "5103716820a4adab0e55cfd59c7535c8"
server_key_iv: "7f9a18b05f9c00f6d8eddc8911cfea4b"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.587873 31045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:48:49.588620 31068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.588786 31045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 18:48:49.588879 31045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "0543a4a16bad4cf0ae0a63ba1278348f"
format_stamp: "Formatted at 2026-08-11 18:48:49 on dist-test-slave-pww0"
server_key: "5103716820a4adab0e55cfd59c7535c8"
server_key_iv: "7f9a18b05f9c00f6d8eddc8911cfea4b"
server_key_version: "encryptionkey@0"
I20260811 18:48:49.588944 31045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:48:49.602077 31045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:48:49.604138 31045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:48:49.604295 31045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:48:49.604533 31045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:48:49.604846 31045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:48:49.604880 31045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.604910 31045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:48:49.604939 31045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:48:49.611006 31045 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:43223
I20260811 18:48:49.611054 31308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:43223 every 8 connection(s)
I20260811 18:48:49.611383 31045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[30315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474129, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:48:49.620460 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.612144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:60019 (local address 127.9.215.62:45867)
0811 18:48:49.612224 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.612227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.612425 (+ 198us) server_negotiation.cc:407] Connection header received
0811 18:48:49.612924 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.612925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.612936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.612987 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:49.613935 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.614439 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.615402 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.615650 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.615715 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.617823 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:49.617824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:49.617825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:49.617833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:49.619096 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.619511 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.619512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.619512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.619567 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.619804 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.619805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.619805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.619918 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:49.619973 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.620275 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.620364 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:48:49.620615 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.612116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45867 (local address 127.9.215.3:60019)
0811 18:48:49.612304 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.612344 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.612843 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.613046 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.613049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.613643 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:48:49.613851 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.613856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.614508 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.614508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.615338 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.615340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.615738 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.615738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.615848 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.616325 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:48:49.616334 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:48:49.617730 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:48:49.619183 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:49.619192 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:49.619195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:49.619413 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:49.619620 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:48:49.619621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:48:49.619621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:48:49.619745 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:48:49.619973 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:48:49.619981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:48:49.620123 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.620221 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.620388 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260811 18:48:49.620991 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 31045
I20260811 18:48:49.621083 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:48:49.621129 31312 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45867
I20260811 18:48:49.621232 31312 heartbeater.cc:464] Registering TS with master...
I20260811 18:48:49.621279 10076 external_mini_cluster.cc:1428] Setting key 7b295b420a8e8781247fe5ffb65f1fe2
I20260811 18:48:49.621435 31312 heartbeater.cc:511] Master 127.9.215.62:45867 requested a full tablet report, sending...
I20260811 18:48:49.621869 30392 ts_manager.cc:194] Registered new tserver with Master: 0543a4a16bad4cf0ae0a63ba1278348f (127.9.215.3:43223)
I20260811 18:48:49.622234 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:48:49.622303 30392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:60019
I20260811 18:48:49.629166 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.623539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.9.215.62:45867)
0811 18:48:49.623595 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.623597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.623698 (+ 101us) server_negotiation.cc:407] Connection header received
0811 18:48:49.623741 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.623742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.623750 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.623789 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:48:49.624335 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.624811 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.625465 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.625645 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.625706 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.626864 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:48:49.626865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:48:49.626865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:48:49.626872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:48:49.628207 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.628518 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.628519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.628519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.628563 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:48:49.628714 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:48:49.628714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:48:49.628715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:48:49.628862 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:48:49.628937 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.629018 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.629088 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:48:49.631021 30392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:48:49.632872 30392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:48:49.639961 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.635817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.9.215.2:34141)
0811 18:48:49.636105 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.636110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.636358 (+ 248us) server_negotiation.cc:407] Connection header received
0811 18:48:49.636422 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.636424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.636437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.636499 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:49.636618 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.637354 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.638711 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.639516 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.639595 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.639666 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.639818 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.639868 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260811 18:48:49.640036 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.635434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43223 (local address 127.0.0.1:51420)
0811 18:48:49.635639 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.635686 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.635757 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.636241 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.636244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.636246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:49.636333 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.636335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.637369 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.637371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.638775 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.638778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.639761 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.639762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.639827 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.639832 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.639879 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.639923 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260811 18:48:49.639943 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.635708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:34141 (local address 127.0.0.1:51988)
0811 18:48:49.636284 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.636294 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.636356 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.636510 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.636511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.636512 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:49.636566 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.636567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.637442 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.637443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.638624 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.638627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.639602 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.639604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.639698 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.639714 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.639792 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.639845 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":82,"thread_start_us":76,"threads_started":1}
I20260811 18:48:49.640125 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.635868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.9.215.3:43223)
0811 18:48:49.636049 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.636056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.636066 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:48:49.636108 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.636110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.636128 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.636218 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:49.636449 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.637276 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.638867 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.639607 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.639664 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.639728 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.639991 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.640038 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260811 18:48:49.640954 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.635438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:34959 (local address 127.0.0.1:35690)
0811 18:48:49.635831 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.635841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.635899 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.637621 (+ 1722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.637622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.637624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:49.637694 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.637696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.638981 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.638982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.639850 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.639852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.640733 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.640734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.640803 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.640807 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.640837 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.640878 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":26,"thread_start_us":141,"threads_started":1}
I20260811 18:48:49.641314 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.636100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.9.215.1:34959)
0811 18:48:49.637461 (+ 1361us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.637465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.637476 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:48:49.637515 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.637517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.637532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.637614 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:49.637755 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.638873 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.639936 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.640634 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.641072 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.641149 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.641176 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.641229 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1317,"thread_start_us":89,"threads_started":1}
I20260811 18:48:49.641256 31178 tablet_service.cc:1467] Processing CreateTablet for tablet d2d786221a1d498d80a053cda10d31f4 (DEFAULT_TABLE table=test-table [id=63cb24f8b2754bdb97bf00c1f47a7580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:49.641278 30890 tablet_service.cc:1467] Processing CreateTablet for tablet d2d786221a1d498d80a053cda10d31f4 (DEFAULT_TABLE table=test-table [id=63cb24f8b2754bdb97bf00c1f47a7580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:49.641624 31178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d786221a1d498d80a053cda10d31f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:49.641635 30890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d786221a1d498d80a053cda10d31f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:49.644399 30641 tablet_service.cc:1467] Processing CreateTablet for tablet d2d786221a1d498d80a053cda10d31f4 (DEFAULT_TABLE table=test-table [id=63cb24f8b2754bdb97bf00c1f47a7580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:48:49.644526 31346 tablet_bootstrap.cc:492] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Bootstrap starting.
I20260811 18:48:49.644798 30641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d786221a1d498d80a053cda10d31f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:48:49.644817 31345 tablet_bootstrap.cc:492] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Bootstrap starting.
I20260811 18:48:49.645177 31346 tablet_bootstrap.cc:654] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:49.645471 31345 tablet_bootstrap.cc:654] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:49.645594 31346 log.cc:824] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:49.646028 31345 log.cc:824] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:49.646296 31346 tablet_bootstrap.cc:492] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: No bootstrap required, opened a new log
I20260811 18:48:49.646363 31346 ts_tablet_manager.cc:1397] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:48:49.646845 31345 tablet_bootstrap.cc:492] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: No bootstrap required, opened a new log
I20260811 18:48:49.646906 31345 ts_tablet_manager.cc:1397] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:49.648092 31346 raft_consensus.cc:348] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.648281 31346 raft_consensus.cc:374] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:49.648322 31346 raft_consensus.cc:729] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de59f06735f645108946728c99fe3ba7, State: Initialized, Role: FOLLOWER
I20260811 18:48:49.648305 31354 tablet_bootstrap.cc:492] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa: Bootstrap starting.
I20260811 18:48:49.648473 31346 consensus_queue.cc:260] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.649011 31354 tablet_bootstrap.cc:654] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa: Neither blocks nor log segments found. Creating new log.
I20260811 18:48:49.649008 31346 ts_tablet_manager.cc:1428] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:48:49.649144 31036 heartbeater.cc:503] Master 127.9.215.62:45867 was elected leader, sending a full tablet report...
I20260811 18:48:49.649151 31345 raft_consensus.cc:348] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.649317 31345 raft_consensus.cc:374] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:49.649385 31345 raft_consensus.cc:729] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0543a4a16bad4cf0ae0a63ba1278348f, State: Initialized, Role: FOLLOWER
I20260811 18:48:49.649515 31345 consensus_queue.cc:260] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.649580 31354 log.cc:824] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa: Log is configured to *not* fsync() on all Append() calls
I20260811 18:48:49.649796 31345 ts_tablet_manager.cc:1428] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:48:49.649981 31312 heartbeater.cc:503] Master 127.9.215.62:45867 was elected leader, sending a full tablet report...
I20260811 18:48:49.650949 31354 tablet_bootstrap.cc:492] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa: No bootstrap required, opened a new log
I20260811 18:48:49.651031 31354 ts_tablet_manager.cc:1397] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:48:49.652513 31354 raft_consensus.cc:348] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.652661 31354 raft_consensus.cc:374] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:48:49.652693 31354 raft_consensus.cc:729] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a983437c1f14c4f99065c64ad6fcbfa, State: Initialized, Role: FOLLOWER
I20260811 18:48:49.652781 31354 consensus_queue.cc:260] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.653014 31354 ts_tablet_manager.cc:1428] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:48:49.653090 30766 heartbeater.cc:503] Master 127.9.215.62:45867 was elected leader, sending a full tablet report...
I20260811 18:48:49.688382 31364 raft_consensus.cc:482] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:48:49.688524 31364 raft_consensus.cc:504] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.688942 31364 leader_election.cc:288] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0543a4a16bad4cf0ae0a63ba1278348f (127.9.215.3:43223), de59f06735f645108946728c99fe3ba7 (127.9.215.2:34141)
I20260811 18:48:49.692952 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.689287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:51411 (local address 127.9.215.2:34141)
0811 18:48:49.689335 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.689337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.689485 (+ 148us) server_negotiation.cc:407] Connection header received
0811 18:48:49.689512 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.689514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.689524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.689568 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:49.689724 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.690413 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.691964 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.692600 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.692672 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.692685 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.692840 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.692889 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:48:49.692961 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.689177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:34141 (local address 127.9.215.1:51411)
0811 18:48:49.689385 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.689395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.689448 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.689580 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.689580 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.689582 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:49.689663 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.689664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.690501 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.690501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.691867 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.691871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.692675 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.692676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.692753 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.692758 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.692806 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.692857 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":6,"thread_start_us":156,"threads_started":1}
I20260811 18:48:49.693007 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.689224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:46139 (local address 127.9.215.3:43223)
0811 18:48:49.689288 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.689290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.689302 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:48:49.689331 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.689332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.689344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.689407 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:48:49.689578 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.690344 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.691964 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.692594 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.692704 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.692715 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.692898 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.692943 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:48:49.693053 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.689113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43223 (local address 127.9.215.1:46139)
0811 18:48:49.689172 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.689185 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:48:49.689235 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:48:49.689426 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:48:49.689427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:48:49.689430 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:48:49.689512 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.689515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.690474 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.690476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.691867 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:48:49.691871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.692731 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.692733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:48:49.692824 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.692827 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:48:49.692875 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:48:49.692923 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":67}
I20260811 18:48:49.693423 31214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d786221a1d498d80a053cda10d31f4" candidate_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" is_pre_election: true
I20260811 18:48:49.693471 30933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d786221a1d498d80a053cda10d31f4" candidate_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de59f06735f645108946728c99fe3ba7" is_pre_election: true
I20260811 18:48:49.693627 31214 raft_consensus.cc:2454] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a983437c1f14c4f99065c64ad6fcbfa in term 0.
I20260811 18:48:49.693627 30933 raft_consensus.cc:2454] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a983437c1f14c4f99065c64ad6fcbfa in term 0.
I20260811 18:48:49.693868 30540 leader_election.cc:302] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a983437c1f14c4f99065c64ad6fcbfa, de59f06735f645108946728c99fe3ba7; no voters:
I20260811 18:48:49.694033 31364 raft_consensus.cc:2781] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:48:49.694109 31364 raft_consensus.cc:482] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:48:49.694126 31364 raft_consensus.cc:3037] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:49.694837 31364 raft_consensus.cc:504] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.694962 31364 leader_election.cc:288] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [CANDIDATE]: Term 1 election: Requested vote from peers 0543a4a16bad4cf0ae0a63ba1278348f (127.9.215.3:43223), de59f06735f645108946728c99fe3ba7 (127.9.215.2:34141)
I20260811 18:48:49.695160 30933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d786221a1d498d80a053cda10d31f4" candidate_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de59f06735f645108946728c99fe3ba7"
I20260811 18:48:49.695150 31214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d786221a1d498d80a053cda10d31f4" candidate_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0543a4a16bad4cf0ae0a63ba1278348f"
I20260811 18:48:49.695240 31214 raft_consensus.cc:3037] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:48:49.695241 30933 raft_consensus.cc:3037] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 0 FOLLOWER]: Advancing to term 1
W20260811 18:48:49.695991 31038 tablet.cc:2369] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:48:49.696210 31214 raft_consensus.cc:2454] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a983437c1f14c4f99065c64ad6fcbfa in term 1.
I20260811 18:48:49.696210 30933 raft_consensus.cc:2454] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a983437c1f14c4f99065c64ad6fcbfa in term 1.
I20260811 18:48:49.696376 30540 leader_election.cc:302] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a983437c1f14c4f99065c64ad6fcbfa, de59f06735f645108946728c99fe3ba7; no voters:
I20260811 18:48:49.696460 31364 raft_consensus.cc:2781] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:48:49.696622 31364 raft_consensus.cc:686] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [term 1 LEADER]: Becoming Leader. State: Replica: 8a983437c1f14c4f99065c64ad6fcbfa, State: Running, Role: LEADER
I20260811 18:48:49.696724 31364 consensus_queue.cc:237] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } }
I20260811 18:48:49.697665 30392 catalog_manager.cc:5495] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a983437c1f14c4f99065c64ad6fcbfa (127.9.215.1). New cstate: current_term: 1 leader_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a983437c1f14c4f99065c64ad6fcbfa" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 } health_report { overall_health: UNKNOWN } } }
I20260811 18:48:49.717758 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.715316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.9.215.1:34959)
0811 18:48:49.715378 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:48:49.715380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:48:49.715388 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:48:49.715430 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:48:49.715432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:48:49.715477 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:48:49.715545 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:48:49.715706 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.716202 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.716947 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:48:49.717155 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:48:49.717217 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:48:49.717336 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:48:49.717455 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:48:49.717598 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:48:49.717626 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 18:48:49.717649 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:48:49.719794 31214 raft_consensus.cc:1261] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 1 FOLLOWER]: Refusing update from remote peer 8a983437c1f14c4f99065c64ad6fcbfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:49.719787 30933 raft_consensus.cc:1261] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 1 FOLLOWER]: Refusing update from remote peer 8a983437c1f14c4f99065c64ad6fcbfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:48:49.720142 31364 consensus_queue.cc:1035] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "de59f06735f645108946728c99fe3ba7" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:49.720305 31366 consensus_queue.cc:1035] T d2d786221a1d498d80a053cda10d31f4 P 8a983437c1f14c4f99065c64ad6fcbfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0543a4a16bad4cf0ae0a63ba1278348f" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:48:49.722774 31369 mvcc.cc:205] Tried to move back new op lower bound from 7317398035329224704 to 7317398035238420480. Current Snapshot: MvccSnapshot[applied={T|T < 7317398035329224704}]
I20260811 18:48:49.723654 31370 mvcc.cc:205] Tried to move back new op lower bound from 7317398035329224704 to 7317398035238420480. Current Snapshot: MvccSnapshot[applied={T|T < 7317398035329224704}]
I20260811 18:48:49.726397 30391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:49.726521 30391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:48:49.727415 30391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8053058249b9466ba8a651fd7d1e8743: Sending DeleteTablet for 3 replicas of tablet d2d786221a1d498d80a053cda10d31f4
I20260811 18:48:49.727815 30641 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d786221a1d498d80a053cda10d31f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:49 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:35690
I20260811 18:48:49.727874 31178 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d786221a1d498d80a053cda10d31f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:49 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:51420
I20260811 18:48:49.727885 30890 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d786221a1d498d80a053cda10d31f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:48:49 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:51988
I20260811 18:48:49.728169 31383 tablet_replica.cc:330] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: stopping tablet replica
I20260811 18:48:49.728330 31383 raft_consensus.cc:2229] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:49.728423 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 30500
I20260811 18:48:49.728456 31383 raft_consensus.cc:2258] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:49.732028 31386 tablet_replica.cc:330] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: stopping tablet replica
I20260811 18:48:49.732188 31386 raft_consensus.cc:2229] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:48:49.732295 31386 raft_consensus.cc:2258] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:48:49.748199 31383 ts_tablet_manager.cc:1905] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:49.750002 31383 ts_tablet_manager.cc:1918] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:49.750154 31386 ts_tablet_manager.cc:1905] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:48:49.750198 31383 log.cc:1196] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/d2d786221a1d498d80a053cda10d31f4
I20260811 18:48:49.750514 31383 ts_tablet_manager.cc:1939] T d2d786221a1d498d80a053cda10d31f4 P 0543a4a16bad4cf0ae0a63ba1278348f: Deleting consensus metadata
I20260811 18:48:49.751070 30377 catalog_manager.cc:4841] TS 0543a4a16bad4cf0ae0a63ba1278348f (127.9.215.3:43223): tablet d2d786221a1d498d80a053cda10d31f4 (table test-table [id=63cb24f8b2754bdb97bf00c1f47a7580]) successfully deleted
I20260811 18:48:49.751973 31386 ts_tablet_manager.cc:1918] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:48:49.752055 31386 log.cc:1196] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/d2d786221a1d498d80a053cda10d31f4
I20260811 18:48:49.752313 31386 ts_tablet_manager.cc:1939] T d2d786221a1d498d80a053cda10d31f4 P de59f06735f645108946728c99fe3ba7: Deleting consensus metadata
I20260811 18:48:49.755654 30372 catalog_manager.cc:4841] TS de59f06735f645108946728c99fe3ba7 (127.9.215.2:34141): tablet d2d786221a1d498d80a053cda10d31f4 (table test-table [id=63cb24f8b2754bdb97bf00c1f47a7580]) successfully deleted
W20260811 18:48:49.761423 30372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:34959 (error 108)
I20260811 18:48:49.761592 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 30780
I20260811 18:48:49.768440 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:48:49.768186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:34959 (local address 127.0.0.1:35704)
0811 18:48:49.768307 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 18:48:49.768384 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:34959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 18:48:49.773664 30372 catalog_manager.cc:4584] TS 8a983437c1f14c4f99065c64ad6fcbfa (127.9.215.1:34959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2d786221a1d498d80a053cda10d31f4: Network error: Client connection negotiation failed: client connection to 127.9.215.1:34959: connect: Connection refused (error 111)
I20260811 18:48:49.780388 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 31045
I20260811 18:48:49.797971 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 30343
2026-08-11T18:48:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2272 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[31396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[31396](info): set up 2 sockets
Aug 11 18:48:49 dist-test-slave-pww0 krb5kdc[31396](info): commencing operation
krb5kdc: starting...
W20260811 18:48:51.045961 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:48:51 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:48:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/rangeradmin_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/rangeradmin_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/rangerlookup_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/rangerlookup_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
I20260811 18:48:51.232061 10076 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdX0PXQ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/postgres -l logfile start
2026-08-11 18:48:53.927 UTC [31450] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:48:53.927 UTC [31450] LOG: listening on IPv4 address "127.9.215.20", port 38251
2026-08-11 18:48:53.930 UTC [31450] LOG: listening on Unix socket "/tmp/.s.PGSQL.38251"
2026-08-11 18:48:53.961 UTC [31455] LOG: database system was shut down at 2026-08-11 18:48:52 UTC
2026-08-11 18:48:53.965 UTC [31450] LOG: database system is ready to accept connections
I20260811 18:48:55.094964 10076 mini_postgres.cc:96] Postgres bound to 38251
2026-08-11 18:48:55.106 UTC [31472] FATAL: database "slave" does not exist
127.9.215.20:38251 - accepting connections
I20260811 18:48:55.106937 10076 mini_ranger.cc:161] Starting Ranger...
I20260811 18:48:55.126310 10076 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:48:55.647380 10076 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:48:55.647498 10076 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-admin
2026-08-11 18:48:56,060 [I] DB FLAVOR :POSTGRES
2026-08-11 18:48:56,061 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:48:56,061 [I] Checking connection..
2026-08-11 18:48:56,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:48:56,362 [I] Checking connection passed.
2026-08-11 18:48:56,363 [I] --------- Verifying version history table ---------
2026-08-11 18:48:56,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:48:56,642 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:48:56,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:48:56,918 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:48:56,919 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:48:56,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:48:57.182 UTC [31598] WARNING: there is no transaction in progress
2026-08-11 18:48:57,189 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:48:57,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:48:57,466 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:48:57,466 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:48:57,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:48:57,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:48:58,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pww0.c.gcp-upstream.internal','N') ;"
2026-08-11 18:48:58,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:48:58,517 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:48:58,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:48:58,778 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:48:58,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:48:59,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:48:59,300 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:48:59,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:49:00.386 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.405 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.424 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.442 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.460 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.483 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.548 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.556 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.572 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.584 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.596 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.612 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.623 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.630 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.638 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:00.646 UTC [31814] WARNING: there is no transaction in progress
2026-08-11 18:49:01,416 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:49:01,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:49:01,690 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:49:01,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:49:01,962 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:49:01,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:49:02,242 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:49:02,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:49:02,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:49:02,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pww0.c.gcp-upstream.internal';"
2026-08-11 18:49:03,029 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:49:03,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:49:03,310 [I] DB_PATCHES have already been applied
I20260811 18:49:03.315831 10076 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdX0PXQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:49:03.315904 10076 mini_ranger.cc:193] Using host: 127.9.215.20
I20260811 18:49:03.319736 10076 mini_ranger.cc:227] Ranger admin URL: http://127.9.215.20:48455
Aug 11, 2026 6:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:49:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:49:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:49:04.668232 10076 mini_ranger.cc:160] Time spent starting Ranger: real 9.561s user 0.000s sys 0.012s
Aug 11, 2026 6:49:26 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:49:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:49:26 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:49:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:49:27 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:49:28.514948 10076 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/rangerkms_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/rangerkms_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.215.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.215.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:49:28 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474168, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:49:28.619446 10076 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:49:28.630281 10076 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:49:29.588573 10076 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:49:29.588653 10076 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms
I20260811 18:49:29.588665 10076 mini_ranger_kms.cc:227] Using postgres at 127.9.215.20:38251
2026-08-11 18:49:30,250 [I] DB FLAVOR :POSTGRES
2026-08-11 18:49:30,250 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:49:30,251 [I] Checking connection
2026-08-11 18:49:30,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:49:30,537 [I] connection success
2026-08-11 18:49:30,538 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:49:30,538 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:49:30,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:49:30,792 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:49:30,792 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:49:30,792 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:49:30,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:38251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:49:31,070 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:49:31.657284 10076 mini_ranger_kms.cc:310] Created kms service
I20260811 18:49:31.859076 10076 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:49:31.906826 10076 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:49:31 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474168, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:49:32.368445 10076 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:49:32.368585 10076 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:49:32.368606 10076 mini_ranger_kms.cc:240] Using host: 127.9.215.20
I20260811 18:49:32.373663 10076 mini_ranger_kms.cc:276] Ranger KMS PID: 32143
I20260811 18:49:32.373804 10076 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.215.20:43987
Aug 11, 2026 6:49:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:49:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:49:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:49:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:49:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:49:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:49:34 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474174, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.215.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:35.069939 10076 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.451s user 0.013s sys 0.348s
I20260811 18:49:35.070060 10076 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:49:35.070137 10076 mini_ranger_kms.cc:401] 127.9.215.20:43987/kms/v1/keys
Aug 11 18:49:35 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474174, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.215.20@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
Aug 11 18:49:35 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474174, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.215.20@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
Aug 11 18:49:38 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:49:38.446864 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36649
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36649
--ranger_config_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:43987/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:38.545279 32240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:38.545507 32240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:38.545531 32240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:38.546737 32240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:49:38.546785 32240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:38.546798 32240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:38.546809 32240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:49:38.546820 32240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:49:38.548679 32240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.215.20:43987/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36649
--ranger_config_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:49:38.548950 32240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:38.549206 32240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:38.555868 32246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:38.555959 32240 server_base.cc:1034] running on GCE node
W20260811 18:49:38.555972 32247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:38.556007 32251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:38.556484 32240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:38.556871 32240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:38.558056 32240 hybrid_clock.cc:648] HybridClock initialized: now 1786474178558028 us; error 41 us; skew 500 ppm
Aug 11 18:49:38 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:38.564548 32240 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:49:38.565192 32240 webserver.cc:460] Webserver started at http://127.9.215.62:33013/ using document root <none> and password file <none>
I20260811 18:49:38.565483 32240 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:38.565548 32240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:38.565716 32240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:49:38 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:49:38.628621 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "36a4bbaef36a419aa29bfbcaacd851a8"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-pww0"
server_key: "aef4489a025f2bf6c4dc5984397b1790"
server_key_iv: "7567a7f2e0e1edb0f6265fd455601ef8"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:38.629180 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "36a4bbaef36a419aa29bfbcaacd851a8"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-pww0"
server_key: "aef4489a025f2bf6c4dc5984397b1790"
server_key_iv: "7567a7f2e0e1edb0f6265fd455601ef8"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:38.631152 32240 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:49:38.649552 32262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:38.649796 32240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:49:38.649940 32240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "36a4bbaef36a419aa29bfbcaacd851a8"
format_stamp: "Formatted at 2026-08-11 18:49:38 on dist-test-slave-pww0"
server_key: "aef4489a025f2bf6c4dc5984397b1790"
server_key_iv: "7567a7f2e0e1edb0f6265fd455601ef8"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:38.650048 32240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:38.677278 32240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:38.679427 32240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:38.679651 32240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:38.684957 32240 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36649
I20260811 18:49:38.684984 32355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36649 every 8 connection(s)
I20260811 18:49:38.685402 32240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:49:38.686300 32362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:38.687156 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 32240
I20260811 18:49:38.687251 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:49:38.687407 10076 external_mini_cluster.cc:1428] Setting key 84de62b0287501dceef673ae13513dba
I20260811 18:49:38.689749 32362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8: Bootstrap starting.
I20260811 18:49:38.690493 32362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:38.690828 32362 log.cc:824] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:38.691694 32362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8: No bootstrap required, opened a new log
Aug 11 18:49:38 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:49:38.694290 32362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36649 } }
I20260811 18:49:38.694478 32362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:38.694514 32362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a4bbaef36a419aa29bfbcaacd851a8, State: Initialized, Role: FOLLOWER
I20260811 18:49:38.694697 32362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36649 } }
I20260811 18:49:38.694784 32362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:49:38.694813 32362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:49:38.694880 32362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:38.695624 32362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36649 } }
I20260811 18:49:38.695736 32362 leader_election.cc:302] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a4bbaef36a419aa29bfbcaacd851a8; no voters:
I20260811 18:49:38.695947 32362 leader_election.cc:288] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:49:38.695997 32367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:38.696213 32367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [term 1 LEADER]: Becoming Leader. State: Replica: 36a4bbaef36a419aa29bfbcaacd851a8, State: Running, Role: LEADER
I20260811 18:49:38.696327 32362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:49:38.696359 32367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36649 } }
I20260811 18:49:38.696949 32367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36649 } } }
I20260811 18:49:38.697161 32367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:38.697428 32367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a4bbaef36a419aa29bfbcaacd851a8. Latest consensus state: current_term: 1 leader_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a4bbaef36a419aa29bfbcaacd851a8" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36649 } } }
I20260811 18:49:38.697573 32367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:49:38.697746 32386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:49:38.698114 32362 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:49:38.698292 32386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:49:38.698958 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:38.688092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.9.215.62:36649)
0811 18:49:38.688270 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:38.688281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:38.688309 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:49:38.688808 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:38.688816 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:38.688883 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:38.689033 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:38.689661 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.690660 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.691600 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:38.692106 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:38.692217 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:38.694572 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:38.694577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:38.694578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:38.694592 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:38.696608 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.697070 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.697074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.697074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.697241 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:38.697574 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:38.697575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:38.697575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:38.697932 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:38.698086 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:38.698629 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 18:49:38.698755 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260811 18:49:38.705039 32386 catalog_manager.cc:1347] Generated new cluster ID: b8c5745c1389405a9ff8f3d43b0156d2
I20260811 18:49:38.705140 32386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:49:38.715049 32386 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:49:38.716013 32386 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:49:38.726706 32386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8: Generated new TSK 0
I20260811 18:49:38.727054 32386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:49:39.504997 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:43987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36649
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:39.624738 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:39.624962 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:39.624997 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:39.626269 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:39.626320 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:39.626358 32438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:49:39.628053 32438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.215.20:43987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:49:39.628324 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:39.628561 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:39.635179 32455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:39.635356 32438 server_base.cc:1034] running on GCE node
W20260811 18:49:39.635181 32459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:39.635510 32456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:39.635773 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:39.636089 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:39.637261 32438 hybrid_clock.cc:648] HybridClock initialized: now 1786474179637245 us; error 35 us; skew 500 ppm
Aug 11 18:49:39 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:39.640648 32438 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:49:39.641330 32438 webserver.cc:460] Webserver started at http://127.9.215.1:42901/ using document root <none> and password file <none>
I20260811 18:49:39.641635 32438 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:39.641710 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:39.641861 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:49:39 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:49:39.658574 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "70134161b1d14aeba557c4d53da6922b"
format_stamp: "Formatted at 2026-08-11 18:49:39 on dist-test-slave-pww0"
server_key: "2eb2aee8645e80e256ed9874b1aa2d1d"
server_key_iv: "6b67d2dbffcb406558541ccf3587979e"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:39.659157 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "70134161b1d14aeba557c4d53da6922b"
format_stamp: "Formatted at 2026-08-11 18:49:39 on dist-test-slave-pww0"
server_key: "2eb2aee8645e80e256ed9874b1aa2d1d"
server_key_iv: "6b67d2dbffcb406558541ccf3587979e"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:39.664012 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260811 18:49:39.685407 32482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:39.685750 32438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260811 18:49:39.685923 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "70134161b1d14aeba557c4d53da6922b"
format_stamp: "Formatted at 2026-08-11 18:49:39 on dist-test-slave-pww0"
server_key: "2eb2aee8645e80e256ed9874b1aa2d1d"
server_key_iv: "6b67d2dbffcb406558541ccf3587979e"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:39.686137 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:39.719969 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:39.722585 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:39.722819 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:39.723136 32438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:39.723600 32438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:39.723690 32438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:39.723783 32438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:39.723858 32438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:39.731302 32438 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:34377
I20260811 18:49:39.731984 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:49:39.733304 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 32438
I20260811 18:49:39.733419 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:49:39.733628 10076 external_mini_cluster.cc:1428] Setting key 049884c24e74aac87cc7b25e9b800737
I20260811 18:49:39.748417 32726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:34377 every 8 connection(s)
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Aug 11 18:49:39 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:49:39.764973 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:39.744281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36649 (local address 127.9.215.1:36839)
0811 18:49:39.748066 (+ 3785us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:39.748157 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:39.748875 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:39.751991 (+ 3116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:39.751998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:39.752271 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:39.752591 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:39.752598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:39.754412 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:39.754413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:39.755838 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:39.755843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:39.756714 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:39.756716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:39.756896 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:39.757542 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:39.757556 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:39.761026 (+ 3470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:39.763176 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:39.763189 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:39.763192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:39.763552 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:39.763767 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:39.763769 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:39.763769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:39.763918 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:39.764210 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:39.764218 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:39.764418 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 18:49:39.764515 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 18:49:39.764693 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3707,"thread_start_us":236,"threads_started":1}
I20260811 18:49:39.765806 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:39.744804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:36839 (local address 127.9.215.62:36649)
0811 18:49:39.751612 (+ 6808us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:39.751614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:39.751629 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:49:39.751661 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:39.751662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:39.751678 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:39.751775 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:39.752716 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:39.754250 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:39.756189 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:39.756525 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:39.756613 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:39.761192 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:39.761194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:39.761195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:39.761205 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:39.763055 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:39.763639 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:39.763641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:39.763641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:39.763708 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:39.763979 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:39.763979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:39.763980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:39.764130 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:39.764184 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:39.765604 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0811 18:49:39.765703 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6776,"thread_start_us":78,"threads_started":1}
I20260811 18:49:39.766234 32728 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36649
I20260811 18:49:39.766345 32728 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:39.766573 32728 heartbeater.cc:511] Master 127.9.215.62:36649 requested a full tablet report, sending...
I20260811 18:49:39.767172 32295 ts_manager.cc:194] Registered new tserver with Master: 70134161b1d14aeba557c4d53da6922b (127.9.215.1:34377)
I20260811 18:49:39.768108 32295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:36839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:49:39.782845 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:43987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36649
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:49:39 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:49:39.915066 32747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:39.915405 32747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:39.915442 32747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:39.917484 32747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:39.917570 32747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:39.917615 32747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:49:39.920953 32747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.215.20:43987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:49:39.921309 32747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:39.921666 32747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:39.927758 32755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:39.927886 32747 server_base.cc:1034] running on GCE node
W20260811 18:49:39.927898 32758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:39.927685 32760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:39.928738 32747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:39.929096 32747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:39.930311 32747 hybrid_clock.cc:648] HybridClock initialized: now 1786474179930280 us; error 46 us; skew 500 ppm
Aug 11 18:49:39 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:39.933266 32747 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:49:39.933779 32747 webserver.cc:460] Webserver started at http://127.9.215.2:33789/ using document root <none> and password file <none>
I20260811 18:49:39.934031 32747 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:39.934144 32747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:39.934309 32747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:49:39 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:49:39.950711 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "1261addc5d5e491fbb2ba5149d6e5f26"
format_stamp: "Formatted at 2026-08-11 18:49:39 on dist-test-slave-pww0"
server_key: "a2a39e675f9486fffd718fd8d4450c51"
server_key_iv: "19ef9caca47ee1010f28efa7f758e325"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:39.951682 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "1261addc5d5e491fbb2ba5149d6e5f26"
format_stamp: "Formatted at 2026-08-11 18:49:39 on dist-test-slave-pww0"
server_key: "a2a39e675f9486fffd718fd8d4450c51"
server_key_iv: "19ef9caca47ee1010f28efa7f758e325"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:39.956017 32747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:49:39.969331 307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:39.969544 32747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:49:39.969672 32747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "1261addc5d5e491fbb2ba5149d6e5f26"
format_stamp: "Formatted at 2026-08-11 18:49:39 on dist-test-slave-pww0"
server_key: "a2a39e675f9486fffd718fd8d4450c51"
server_key_iv: "19ef9caca47ee1010f28efa7f758e325"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:39.969774 32747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:40.001559 32747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:40.004369 32747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:40.004585 32747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:40.004967 32747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:40.005409 32747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:40.005465 32747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.005507 32747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:40.005533 32747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.013849 32747 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:34237
I20260811 18:49:40.014369 32747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:49:40.015043 539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:34237 every 8 connection(s)
Aug 11 18:49:40 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:49:40.022420 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 32747
I20260811 18:49:40.022534 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:49:40.022733 10076 external_mini_cluster.cc:1428] Setting key 8889b44d75beacd5d75ba5f2fe6f267b
I20260811 18:49:40.026639 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.015581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:40647 (local address 127.9.215.62:36649)
0811 18:49:40.015633 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.015635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.015648 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:49:40.016372 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.016374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.016387 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.016459 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:40.017354 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.017941 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.019713 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.020052 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.020134 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.023012 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:40.023014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:40.023015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:40.023025 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:40.024893 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.025521 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.025522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.025523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.025601 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.025922 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.025924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.025924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.026053 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:40.026115 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.026433 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.026544 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:49:40.027022 544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.015248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36649 (local address 127.9.215.2:40647)
0811 18:49:40.015446 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.015523 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.016218 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.016618 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.016623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.016899 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:40.017226 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.017235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.018068 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.018069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.019587 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.019591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.020155 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.020156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.020365 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.021049 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:40.021061 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:40.022862 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:40.025009 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.025023 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.025026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.025411 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.025684 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.025686 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.025686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.025855 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.026115 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:40.026124 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:40.026302 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.026402 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.026569 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260811 18:49:40.027683 540 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36649
I20260811 18:49:40.027786 540 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:40.027997 540 heartbeater.cc:511] Master 127.9.215.62:36649 requested a full tablet report, sending...
I20260811 18:49:40.028465 32295 ts_manager.cc:194] Registered new tserver with Master: 1261addc5d5e491fbb2ba5149d6e5f26 (127.9.215.2:34237)
I20260811 18:49:40.028914 32295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:40647
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:49:40.058914 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:43987/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36649
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:39459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:49:40.164610 552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:49:40.164954 552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:49:40.164995 552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:49:40.166923 552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:49:40.167009 552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:49:40.167052 552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:49:40.169540 552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:39459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.215.20:43987/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:36649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:49:40.169864 552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:49:40.170169 552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:49:40.176460 570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:40.176447 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:49:40.176447 567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:49:40.177279 552 server_base.cc:1034] running on GCE node
I20260811 18:49:40.177502 552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:49:40.177765 552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:49:40.178920 552 hybrid_clock.cc:648] HybridClock initialized: now 1786474180178906 us; error 34 us; skew 500 ppm
Aug 11 18:49:40 dist-test-slave-pww0 krb5kdc[31396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:49:40.181879 552 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:49:40.182358 552 webserver.cc:460] Webserver started at http://127.9.215.3:34199/ using document root <none> and password file <none>
I20260811 18:49:40.182570 552 fs_manager.cc:359] Metadata directory not provided
I20260811 18:49:40.182631 552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:49:40.182780 552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:49:40 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:49:40.208717 552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "48ef25e1606a446ab51dc19dc7343233"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-pww0"
server_key: "2a22b8d3cd3532dcf600ebed81f09317"
server_key_iv: "6413043b028f88284e615dfcd1201bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:40.209314 552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "48ef25e1606a446ab51dc19dc7343233"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-pww0"
server_key: "2a22b8d3cd3532dcf600ebed81f09317"
server_key_iv: "6413043b028f88284e615dfcd1201bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:40.211123 552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260811 18:49:40.225677 585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.225874 552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:49:40.225984 552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "48ef25e1606a446ab51dc19dc7343233"
format_stamp: "Formatted at 2026-08-11 18:49:40 on dist-test-slave-pww0"
server_key: "2a22b8d3cd3532dcf600ebed81f09317"
server_key_iv: "6413043b028f88284e615dfcd1201bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:49:40.226090 552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:49:40.251122 552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:49:40.253715 552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:49:40.253888 552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:49:40.254184 552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:49:40.254576 552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:49:40.254619 552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.254649 552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:49:40.254671 552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:49:40.265793 552 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:43457
I20260811 18:49:40.266178 826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:43457 every 8 connection(s)
I20260811 18:49:40.266362 552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:49:40.271584 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 552
I20260811 18:49:40.271705 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:49:40.271888 10076 external_mini_cluster.cc:1428] Setting key 000892f9e71f18f6dc2ac1c7abdab93d
Aug 11 18:49:40 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474180, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:49:40.277220 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.267216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:60255 (local address 127.9.215.62:36649)
0811 18:49:40.267268 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.267271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.267587 (+ 316us) server_negotiation.cc:407] Connection header received
0811 18:49:40.268188 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.268190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.268203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.268239 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:40.268981 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.269731 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.271016 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.271276 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.271350 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.274396 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:40.274398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:40.274398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:40.274406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:40.275827 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.276242 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.276243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.276244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.276303 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.276549 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.276550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.276551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.276678 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:40.276736 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.277035 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.277122 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:49:40.277418 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.267218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:36649 (local address 127.9.215.3:60255)
0811 18:49:40.267416 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.267485 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.268044 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.268453 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.268458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.268678 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:49:40.268884 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.268888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.269857 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.269858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.270919 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.270923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.272072 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.272073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.272243 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.272907 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:49:40.272920 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:49:40.274287 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:49:40.275904 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.275914 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.275917 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.276172 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.276358 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:49:40.276358 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:49:40.276359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:49:40.276489 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:49:40.276734 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:49:40.276742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:49:40.276911 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.277004 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.277180 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260811 18:49:40.277993 828 heartbeater.cc:347] Connected to a master server at 127.9.215.62:36649
I20260811 18:49:40.278107 828 heartbeater.cc:464] Registering TS with master...
I20260811 18:49:40.278306 828 heartbeater.cc:511] Master 127.9.215.62:36649 requested a full tablet report, sending...
I20260811 18:49:40.278697 32295 ts_manager.cc:194] Registered new tserver with Master: 48ef25e1606a446ab51dc19dc7343233 (127.9.215.3:43457)
I20260811 18:49:40.279062 32295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:60255
I20260811 18:49:40.283844 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:49:40.295002 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.286374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.9.215.62:36649)
0811 18:49:40.286424 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.286426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.286709 (+ 283us) server_negotiation.cc:407] Connection header received
0811 18:49:40.286773 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.286774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.286787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.286852 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:49:40.287718 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.288446 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.289504 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.289869 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.289945 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.291664 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:49:40.291665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:49:40.291666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:49:40.291674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:49:40.293448 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.294066 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.294067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.294068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.294140 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:49:40.294343 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:49:40.294344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:49:40.294345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:49:40.294493 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:49:40.294598 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.294813 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.294900 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:49:40.301661 32295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:49:40.304091 32295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:49:40.314608 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.308169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43457 (local address 127.0.0.1:55016)
0811 18:49:40.309170 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.309222 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.309299 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.309862 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.309863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.309866 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:40.309918 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.309919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.311600 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.311601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.313091 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.313094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.314263 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.314264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.314361 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.314377 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.314452 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.314511 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":93,"threads_started":1}
I20260811 18:49:40.315163 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.308501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55016 (local address 127.9.215.3:43457)
0811 18:49:40.309621 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.309625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.309636 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:49:40.309677 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.309679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.309696 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.309772 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:40.310331 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.311419 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.313223 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.314062 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.314135 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.314224 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.315014 (+ 790us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.315075 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":88,"threads_started":1}
I20260811 18:49:40.316084 693 tablet_service.cc:1467] Processing CreateTablet for tablet 5f8012a741cf4376a1ce62d85c6d3b9b (DEFAULT_TABLE table=test-table [id=dfbefe48cecb4560b6f220a896611139]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:40.316510 693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f8012a741cf4376a1ce62d85c6d3b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:40.317884 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.308192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:34237 (local address 127.0.0.1:34644)
0811 18:49:40.309424 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.309433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.309491 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.309804 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.309808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.309811 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:40.309888 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.309891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.314191 (+ 4300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.314192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.315346 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.315349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.317632 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.317633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.317702 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.317707 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.317757 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.317806 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"thread_start_us":124,"threads_started":1}
I20260811 18:49:40.319307 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.308453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:34377 (local address 127.0.0.1:46530)
0811 18:49:40.310017 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.310026 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.310239 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.311549 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.311550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.311552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:40.311602 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.311603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.312782 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.312783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.316005 (+ 3222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.316008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.319058 (+ 3050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.319059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.319135 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.319139 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.319185 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.319231 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"thread_start_us":62,"threads_started":1}
I20260811 18:49:40.319669 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.308783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.9.215.2:34237)
0811 18:49:40.309096 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.309102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.309508 (+ 406us) server_negotiation.cc:407] Connection header received
0811 18:49:40.309573 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.309576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.309594 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.309688 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:40.309961 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.314096 (+ 4135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.316137 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.317406 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.317489 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.317588 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.319517 (+ 1929us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.319581 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":90,"threads_started":1}
I20260811 18:49:40.320518 408 tablet_service.cc:1467] Processing CreateTablet for tablet 5f8012a741cf4376a1ce62d85c6d3b9b (DEFAULT_TABLE table=test-table [id=dfbefe48cecb4560b6f220a896611139]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:40.320928 408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f8012a741cf4376a1ce62d85c6d3b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:40.321828 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.308951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.9.215.1:34377)
0811 18:49:40.309387 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.309392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.311296 (+ 1904us) server_negotiation.cc:407] Connection header received
0811 18:49:40.311341 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.311344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.311361 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.311442 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:40.311670 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.312669 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.318061 (+ 5392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.318883 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.318959 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.319017 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.321681 (+ 2664us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.321750 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":47,"threads_started":1}
I20260811 18:49:40.322599 865 tablet_bootstrap.cc:492] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Bootstrap starting.
I20260811 18:49:40.322778 32579 tablet_service.cc:1467] Processing CreateTablet for tablet 5f8012a741cf4376a1ce62d85c6d3b9b (DEFAULT_TABLE table=test-table [id=dfbefe48cecb4560b6f220a896611139]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:49:40.323308 32579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f8012a741cf4376a1ce62d85c6d3b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:49:40.324156 865 tablet_bootstrap.cc:654] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:40.324620 865 log.cc:824] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:40.325181 867 tablet_bootstrap.cc:492] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Bootstrap starting.
I20260811 18:49:40.326371 867 tablet_bootstrap.cc:654] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:40.326735 867 log.cc:824] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:40.327682 865 tablet_bootstrap.cc:492] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: No bootstrap required, opened a new log
I20260811 18:49:40.327772 865 ts_tablet_manager.cc:1397] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:49:40.327833 867 tablet_bootstrap.cc:492] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: No bootstrap required, opened a new log
I20260811 18:49:40.327903 867 ts_tablet_manager.cc:1397] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:49:40.330092 865 raft_consensus.cc:348] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.330045 867 raft_consensus.cc:348] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.330294 865 raft_consensus.cc:374] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:40.330294 867 raft_consensus.cc:374] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:40.330333 867 raft_consensus.cc:729] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1261addc5d5e491fbb2ba5149d6e5f26, State: Initialized, Role: FOLLOWER
I20260811 18:49:40.330333 865 raft_consensus.cc:729] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ef25e1606a446ab51dc19dc7343233, State: Initialized, Role: FOLLOWER
I20260811 18:49:40.330505 867 consensus_queue.cc:260] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.330505 865 consensus_queue.cc:260] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.330801 867 ts_tablet_manager.cc:1428] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:49:40.330801 865 ts_tablet_manager.cc:1428] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:49:40.330931 540 heartbeater.cc:503] Master 127.9.215.62:36649 was elected leader, sending a full tablet report...
I20260811 18:49:40.330931 828 heartbeater.cc:503] Master 127.9.215.62:36649 was elected leader, sending a full tablet report...
I20260811 18:49:40.332441 873 tablet_bootstrap.cc:492] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: Bootstrap starting.
I20260811 18:49:40.333230 873 tablet_bootstrap.cc:654] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: Neither blocks nor log segments found. Creating new log.
I20260811 18:49:40.333542 873 log.cc:824] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:49:40.334627 873 tablet_bootstrap.cc:492] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: No bootstrap required, opened a new log
I20260811 18:49:40.334698 873 ts_tablet_manager.cc:1397] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:49:40.336812 873 raft_consensus.cc:348] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.336973 873 raft_consensus.cc:374] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:49:40.337008 873 raft_consensus.cc:729] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70134161b1d14aeba557c4d53da6922b, State: Initialized, Role: FOLLOWER
I20260811 18:49:40.337363 873 consensus_queue.cc:260] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.337751 32728 heartbeater.cc:503] Master 127.9.215.62:36649 was elected leader, sending a full tablet report...
I20260811 18:49:40.337738 873 ts_tablet_manager.cc:1428] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
Aug 11 18:49:40 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
W20260811 18:49:40.495620 32729 tablet.cc:2369] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:40.515391 542 tablet.cc:2369] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:49:40.517186 829 tablet.cc:2369] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:49:40.619484 884 raft_consensus.cc:482] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:49:40.619653 884 raft_consensus.cc:504] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.620064 884 leader_election.cc:288] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48ef25e1606a446ab51dc19dc7343233 (127.9.215.3:43457), 1261addc5d5e491fbb2ba5149d6e5f26 (127.9.215.2:34237)
I20260811 18:49:40.624428 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.620619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:48987 (local address 127.9.215.2:34237)
0811 18:49:40.620659 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.620660 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.620864 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:49:40.620984 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.620986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.620999 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.621029 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:40.621269 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.622042 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.623504 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.624057 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.624099 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.624111 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.624308 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.624351 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:49:40.624701 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.620239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:34237 (local address 127.9.215.1:48987)
0811 18:49:40.620781 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.620793 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.620897 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.621103 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.621104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.621106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:40.621192 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.621194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.622135 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.622136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.623365 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.623368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.624164 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.624165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.624233 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.624236 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.624567 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.624614 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":115,"threads_started":1}
I20260811 18:49:40.625058 413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f8012a741cf4376a1ce62d85c6d3b9b" candidate_uuid: "70134161b1d14aeba557c4d53da6922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" is_pre_election: true
I20260811 18:49:40.625219 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.620245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43457 (local address 127.9.215.1:54967)
0811 18:49:40.621014 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.621024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.621104 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.621308 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:49:40.621309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:49:40.621311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:49:40.621373 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.621374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.622848 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.622849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.623930 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:49:40.623932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.624988 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.624988 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:49:40.625047 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.625050 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:49:40.625092 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 18:49:40.625128 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":154,"threads_started":1}
I20260811 18:49:40.625229 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.620523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:54967 (local address 127.9.215.3:43457)
0811 18:49:40.620569 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.620572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.621077 (+ 505us) server_negotiation.cc:407] Connection header received
0811 18:49:40.621175 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.621176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.621189 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.621217 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:49:40.621442 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.622126 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.624031 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.624567 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.624611 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.624622 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.625120 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.625168 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:49:40.625262 413 raft_consensus.cc:2454] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70134161b1d14aeba557c4d53da6922b in term 0.
I20260811 18:49:40.625587 732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f8012a741cf4376a1ce62d85c6d3b9b" candidate_uuid: "70134161b1d14aeba557c4d53da6922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48ef25e1606a446ab51dc19dc7343233" is_pre_election: true
I20260811 18:49:40.625615 32489 leader_election.cc:302] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1261addc5d5e491fbb2ba5149d6e5f26, 70134161b1d14aeba557c4d53da6922b; no voters:
I20260811 18:49:40.625741 732 raft_consensus.cc:2454] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70134161b1d14aeba557c4d53da6922b in term 0.
I20260811 18:49:40.625762 884 raft_consensus.cc:2781] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:49:40.625805 884 raft_consensus.cc:482] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:49:40.625826 884 raft_consensus.cc:3037] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:40.626775 884 raft_consensus.cc:504] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.626931 884 leader_election.cc:288] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [CANDIDATE]: Term 1 election: Requested vote from peers 48ef25e1606a446ab51dc19dc7343233 (127.9.215.3:43457), 1261addc5d5e491fbb2ba5149d6e5f26 (127.9.215.2:34237)
I20260811 18:49:40.627111 732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f8012a741cf4376a1ce62d85c6d3b9b" candidate_uuid: "70134161b1d14aeba557c4d53da6922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48ef25e1606a446ab51dc19dc7343233"
I20260811 18:49:40.627163 413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f8012a741cf4376a1ce62d85c6d3b9b" candidate_uuid: "70134161b1d14aeba557c4d53da6922b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1261addc5d5e491fbb2ba5149d6e5f26"
I20260811 18:49:40.627197 732 raft_consensus.cc:3037] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:40.627244 413 raft_consensus.cc:3037] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:49:40.628162 413 raft_consensus.cc:2454] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70134161b1d14aeba557c4d53da6922b in term 1.
I20260811 18:49:40.628244 732 raft_consensus.cc:2454] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70134161b1d14aeba557c4d53da6922b in term 1.
I20260811 18:49:40.628365 32489 leader_election.cc:302] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1261addc5d5e491fbb2ba5149d6e5f26, 70134161b1d14aeba557c4d53da6922b; no voters:
I20260811 18:49:40.628450 884 raft_consensus.cc:2781] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:49:40.628639 884 raft_consensus.cc:686] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [term 1 LEADER]: Becoming Leader. State: Replica: 70134161b1d14aeba557c4d53da6922b, State: Running, Role: LEADER
I20260811 18:49:40.628767 884 consensus_queue.cc:237] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } }
I20260811 18:49:40.629714 32294 catalog_manager.cc:5495] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b reported cstate change: term changed from 0 to 1, leader changed from <none> to 70134161b1d14aeba557c4d53da6922b (127.9.215.1). New cstate: current_term: 1 leader_uuid: "70134161b1d14aeba557c4d53da6922b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70134161b1d14aeba557c4d53da6922b" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 34377 } health_report { overall_health: HEALTHY } } }
Aug 11 18:49:40 dist-test-slave-pww0 krb5kdc[31396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474179, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:49:40.672977 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.665119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.9.215.1:34377)
0811 18:49:40.665168 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:49:40.665170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:49:40.665179 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:49:40.665227 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:49:40.665228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:49:40.665254 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:49:40.665346 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:49:40.665501 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.666185 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.671575 (+ 5390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:49:40.671924 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:49:40.672017 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:49:40.672311 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:49:40.672472 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:49:40.672553 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:49:40.672839 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 18:49:40.672866 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:49:40.675055 413 raft_consensus.cc:1261] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 1 FOLLOWER]: Refusing update from remote peer 70134161b1d14aeba557c4d53da6922b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:40.675415 732 raft_consensus.cc:1261] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 1 FOLLOWER]: Refusing update from remote peer 70134161b1d14aeba557c4d53da6922b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:49:40.675773 884 consensus_queue.cc:1035] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [LEADER]: Connected to new peer: Peer: permanent_uuid: "48ef25e1606a446ab51dc19dc7343233" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:40.676031 884 consensus_queue.cc:1035] T 5f8012a741cf4376a1ce62d85c6d3b9b P 70134161b1d14aeba557c4d53da6922b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1261addc5d5e491fbb2ba5149d6e5f26" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:49:40.680087 894 mvcc.cc:205] Tried to move back new op lower bound from 7317398244042141696 to 7317398243856076800. Current Snapshot: MvccSnapshot[applied={T|T < 7317398244042141696}]
I20260811 18:49:40.681596 900 mvcc.cc:205] Tried to move back new op lower bound from 7317398244042141696 to 7317398243856076800. Current Snapshot: MvccSnapshot[applied={T|T < 7317398244042141696}]
I20260811 18:49:40.682237 896 mvcc.cc:205] Tried to move back new op lower bound from 7317398244042141696 to 7317398243856076800. Current Snapshot: MvccSnapshot[applied={T|T < 7317398244042141696}]
I20260811 18:49:40.685770 32294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:40.685883 32294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:49:40.686592 32294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a4bbaef36a419aa29bfbcaacd851a8: Sending DeleteTablet for 3 replicas of tablet 5f8012a741cf4376a1ce62d85c6d3b9b
I20260811 18:49:40.687002 408 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f8012a741cf4376a1ce62d85c6d3b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:40 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:34644
I20260811 18:49:40.687328 32579 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f8012a741cf4376a1ce62d85c6d3b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:40 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:46530
I20260811 18:49:40.687772 693 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f8012a741cf4376a1ce62d85c6d3b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:49:40 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:55016
I20260811 18:49:40.688469 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 32438
I20260811 18:49:40.723757 914 tablet_replica.cc:330] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: stopping tablet replica
I20260811 18:49:40.723951 914 raft_consensus.cc:2229] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:40.724255 914 raft_consensus.cc:2258] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:49:40.727622 912 tablet_replica.cc:330] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: stopping tablet replica
I20260811 18:49:40.727806 912 raft_consensus.cc:2229] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:49:40.727954 912 raft_consensus.cc:2258] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:49:40.730230 32263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:34377 (error 108)
I20260811 18:49:40.730882 912 ts_tablet_manager.cc:1905] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:40.731194 914 ts_tablet_manager.cc:1905] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:49:40.732733 914 ts_tablet_manager.cc:1918] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:49:40.732790 912 ts_tablet_manager.cc:1918] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:49:40.732877 912 log.cc:1196] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/5f8012a741cf4376a1ce62d85c6d3b9b
I20260811 18:49:40.732993 914 log.cc:1196] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/5f8012a741cf4376a1ce62d85c6d3b9b
I20260811 18:49:40.733253 914 ts_tablet_manager.cc:1939] T 5f8012a741cf4376a1ce62d85c6d3b9b P 48ef25e1606a446ab51dc19dc7343233: Deleting consensus metadata
I20260811 18:49:40.733273 912 ts_tablet_manager.cc:1939] T 5f8012a741cf4376a1ce62d85c6d3b9b P 1261addc5d5e491fbb2ba5149d6e5f26: Deleting consensus metadata
I20260811 18:49:40.733712 32266 catalog_manager.cc:4841] TS 48ef25e1606a446ab51dc19dc7343233 (127.9.215.3:43457): tablet 5f8012a741cf4376a1ce62d85c6d3b9b (table test-table [id=dfbefe48cecb4560b6f220a896611139]) successfully deleted
I20260811 18:49:40.733721 32267 catalog_manager.cc:4841] TS 1261addc5d5e491fbb2ba5149d6e5f26 (127.9.215.2:34237): tablet 5f8012a741cf4376a1ce62d85c6d3b9b (table test-table [id=dfbefe48cecb4560b6f220a896611139]) successfully deleted
W20260811 18:49:40.733932 858 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:49:40.730754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:34377 (local address 127.0.0.1:46560)
0811 18:49:40.730800 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.730812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:49:40.730859 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:49:40.733888 (+ 3029us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:34377: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":21}
W20260811 18:49:40.734025 32263 catalog_manager.cc:4584] TS 70134161b1d14aeba557c4d53da6922b (127.9.215.1:34377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f8012a741cf4376a1ce62d85c6d3b9b: Network error: Client connection negotiation failed: client connection to 127.9.215.1:34377: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:49:40.734103 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 32747
I20260811 18:49:40.776584 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 552
I20260811 18:49:40.786736 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.786503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:34377 (local address 127.0.0.1:46564)
0811 18:49:40.786597 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.786680 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 18:49:40.792378 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:49:40.792213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:34377 (local address 127.0.0.1:46566)
0811 18:49:40.792284 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 18:49:40.792332 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 18:49:40.796502 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 32240
I20260811 18:49:40.820575 10076 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:49:41.168059 10076 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:49:41.168187 10076 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:49:41.526870 10076 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:49:41.527 UTC [31450] LOG: received smart shutdown request
2026-08-11 18:49:41.531 UTC [31450] LOG: background worker "logical replication launcher" (PID 31468) exited with exit code 1
2026-08-11 18:49:41.531 UTC [31456] LOG: shutting down
2026-08-11 18:49:41.712 UTC [31450] LOG: database system is shut down
2026-08-11T18:49:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51990 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:49:41 dist-test-slave-pww0 krb5kdc[933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:49:41 dist-test-slave-pww0 krb5kdc[933](info): set up 2 sockets
Aug 11 18:49:41 dist-test-slave-pww0 krb5kdc[933](info): commencing operation
krb5kdc: starting...
W20260811 18:49:43.035938 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:49:43 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:49:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/rangeradmin_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/rangeradmin_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/rangerlookup_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/rangerlookup_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
I20260811 18:49:43.234335 10076 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdX0PXQ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/postgres -l logfile start
2026-08-11 18:49:45.854 UTC [990] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:49:45.854 UTC [990] LOG: listening on IPv4 address "127.9.215.20", port 52913
2026-08-11 18:49:45.858 UTC [990] LOG: listening on Unix socket "/tmp/.s.PGSQL.52913"
2026-08-11 18:49:45.887 UTC [999] LOG: database system was shut down at 2026-08-11 18:49:44 UTC
2026-08-11 18:49:45.891 UTC [990] LOG: database system is ready to accept connections
I20260811 18:49:47.025596 10076 mini_postgres.cc:96] Postgres bound to 52913
2026-08-11 18:49:47.033 UTC [1023] FATAL: database "slave" does not exist
127.9.215.20:52913 - accepting connections
I20260811 18:49:47.034086 10076 mini_ranger.cc:161] Starting Ranger...
I20260811 18:49:47.053434 10076 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:49:47.591797 10076 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:49:47.591871 10076 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-admin
2026-08-11 18:49:47,882 [I] DB FLAVOR :POSTGRES
2026-08-11 18:49:47,883 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:49:47,883 [I] Checking connection..
2026-08-11 18:49:47,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:49:48,168 [I] Checking connection passed.
2026-08-11 18:49:48,169 [I] --------- Verifying version history table ---------
2026-08-11 18:49:48,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:49:48,427 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:49:48,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:49:48,707 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:49:48,708 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:49:48,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:49:48.985 UTC [1154] WARNING: there is no transaction in progress
2026-08-11 18:49:48,992 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:49:48,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:49:49,264 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:49:49,264 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:49:49,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:49:49,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:49:49,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pww0.c.gcp-upstream.internal','N') ;"
2026-08-11 18:49:50,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:49:50,323 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:49:50,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:49:50,586 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:49:50,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:49:50,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:49:51,143 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:49:51,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:49:52.214 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.234 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.253 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.272 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.296 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.317 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.383 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.391 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.407 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.420 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.431 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.446 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.457 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.465 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.473 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:52.480 UTC [1374] WARNING: there is no transaction in progress
2026-08-11 18:49:53,296 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:49:53,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:49:53,570 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:49:53,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:49:53,823 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:49:53,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:49:54,110 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:49:54,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:49:54,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:49:54,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pww0.c.gcp-upstream.internal';"
2026-08-11 18:49:54,914 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:49:54,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:49:55,188 [I] DB_PATCHES have already been applied
I20260811 18:49:55.194372 10076 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdX0PXQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:49:55.194445 10076 mini_ranger.cc:193] Using host: 127.9.215.20
I20260811 18:49:55.198693 10076 mini_ranger.cc:227] Ranger admin URL: http://127.9.215.20:47147
Aug 11, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:49:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:49:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:49:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:49:56.558218 10076 mini_ranger.cc:160] Time spent starting Ranger: real 9.524s user 0.000s sys 0.011s
Aug 11, 2026 6:50:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:50:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:50:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:50:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:50:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:50:19.951776 10076 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.215.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.215.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/rangerkms_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.215.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/rangerkms_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.215.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.215.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.215.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/HTTP_127.9.215.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:50:20 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474220, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:50:20.060230 10076 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:50:20.070897 10076 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:50:21.043242 10076 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:50:21.043331 10076 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms
I20260811 18:50:21.043349 10076 mini_ranger_kms.cc:227] Using postgres at 127.9.215.20:52913
2026-08-11 18:50:21,215 [I] DB FLAVOR :POSTGRES
2026-08-11 18:50:21,216 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:50:21,216 [I] Checking connection
2026-08-11 18:50:21,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:50:21,495 [I] connection success
2026-08-11 18:50:21,495 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:50:21,495 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:50:21,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:50:21,755 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:50:21,755 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:50:21,755 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:50:21,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.215.20:52913/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:50:22,075 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:50:22.646752 10076 mini_ranger_kms.cc:310] Created kms service
I20260811 18:50:22.843948 10076 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:50:22.892140 10076 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:50:22 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474220, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:23.287408 10076 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:50:23.287634 10076 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdX0PXQ/build/release/bin/postgresql.jar:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdX0PXQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:50:23.287653 10076 mini_ranger_kms.cc:240] Using host: 127.9.215.20
I20260811 18:50:23.292809 10076 mini_ranger_kms.cc:276] Ranger KMS PID: 1736
I20260811 18:50:23.292896 10076 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.215.20:38867
Aug 11, 2026 6:50:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:50:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:50:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:50:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:50:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:50:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:50:25 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474225, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.215.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:26.015798 10076 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.956s user 0.015s sys 0.141s
I20260811 18:50:26.015913 10076 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:50:26.015962 10076 mini_ranger_kms.cc:401] 127.9.215.20:38867/kms/v1/keys
Aug 11 18:50:26 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474225, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.215.20@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
Aug 11 18:50:26 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474225, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.215.20@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
Aug 11 18:50:28 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:50:28.984081 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40671
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:40671
--ranger_config_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:29.071528 1829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:29.071776 1829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:29.071810 1829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:29.073071 1829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:29.073123 1829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:29.073143 1829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:29.073158 1829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:29.073172 1829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:29.074913 1829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.215.20:38867/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:40671
--ranger_config_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:29.075201 1829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:29.075431 1829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:29.081307 1836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:29.081456 1829 server_base.cc:1034] running on GCE node
W20260811 18:50:29.081360 1838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:29.081329 1841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:29.081887 1829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:29.082265 1829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:29.083500 1829 hybrid_clock.cc:648] HybridClock initialized: now 1786474229083442 us; error 48 us; skew 500 ppm
Aug 11 18:50:29 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:29.090942 1829 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:29.091626 1829 webserver.cc:460] Webserver started at http://127.9.215.62:46029/ using document root <none> and password file <none>
I20260811 18:50:29.091898 1829 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:29.091951 1829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:29.092089 1829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:50:29 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:29.151543 1829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "a02b960edd824c97bdd95f4ee59245a5"
format_stamp: "Formatted at 2026-08-11 18:50:29 on dist-test-slave-pww0"
server_key: "16523a9e811cddec4cd7019fce444f7c"
server_key_iv: "672e13ab5e21355c3d6b9ed6f6385479"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:29.152084 1829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "a02b960edd824c97bdd95f4ee59245a5"
format_stamp: "Formatted at 2026-08-11 18:50:29 on dist-test-slave-pww0"
server_key: "16523a9e811cddec4cd7019fce444f7c"
server_key_iv: "672e13ab5e21355c3d6b9ed6f6385479"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:29.153800 1829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:50:29.171290 1852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:29.171578 1829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:29.171706 1829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "a02b960edd824c97bdd95f4ee59245a5"
format_stamp: "Formatted at 2026-08-11 18:50:29 on dist-test-slave-pww0"
server_key: "16523a9e811cddec4cd7019fce444f7c"
server_key_iv: "672e13ab5e21355c3d6b9ed6f6385479"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:29.171800 1829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:29.218506 1829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:29.220593 1829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:29.220752 1829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:29.225718 1829 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:40671
I20260811 18:50:29.225796 1951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:40671 every 8 connection(s)
I20260811 18:50:29.226176 1829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:29.226936 1955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:29.229674 1955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Bootstrap starting.
I20260811 18:50:29.230340 1955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:29.230641 1955 log.cc:824] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:29.231367 1955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: No bootstrap required, opened a new log
I20260811 18:50:29.232321 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 1829
I20260811 18:50:29.232438 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:50:29.232630 10076 external_mini_cluster.cc:1428] Setting key 3c7810b4ab36f7c666fd2bb5e46e6556
I20260811 18:50:29.234042 1955 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:29.234326 1955 raft_consensus.cc:374] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:29.234383 1955 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02b960edd824c97bdd95f4ee59245a5, State: Initialized, Role: FOLLOWER
I20260811 18:50:29.234541 1955 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:29.234627 1955 raft_consensus.cc:388] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:29.234681 1955 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:29.234740 1955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:29.235512 1955 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:29.235646 1955 leader_election.cc:302] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a02b960edd824c97bdd95f4ee59245a5; no voters:
I20260811 18:50:29.235836 1955 leader_election.cc:288] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:29.235899 1964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:29.236131 1964 raft_consensus.cc:686] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 LEADER]: Becoming Leader. State: Replica: a02b960edd824c97bdd95f4ee59245a5, State: Running, Role: LEADER
I20260811 18:50:29.236240 1955 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:29.236351 1964 consensus_queue.cc:237] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:29.236749 1964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a02b960edd824c97bdd95f4ee59245a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } } }
I20260811 18:50:29.236896 1964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:29.237073 1964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02b960edd824c97bdd95f4ee59245a5. Latest consensus state: current_term: 1 leader_uuid: "a02b960edd824c97bdd95f4ee59245a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } } }
I20260811 18:50:29.237176 1964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:29.237363 1986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:29.237743 1955 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:50:29.237986 1986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:29.240360 2002 mvcc.cc:205] Tried to move back new op lower bound from 7317398442963730432 to 7317398442952654848. Current Snapshot: MvccSnapshot[applied={T|T < 7317398442963730432}]
Aug 11 18:50:29 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:29.247026 1986 catalog_manager.cc:1347] Generated new cluster ID: 679764d61aa04d438d3890158463b295
I20260811 18:50:29.247239 1986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:29.248492 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:29.233350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.9.215.62:40671)
0811 18:50:29.233547 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:29.233559 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:29.233601 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:50:29.234000 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:29.234010 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:29.234083 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:29.234250 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:29.235171 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.236611 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.237753 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:29.238293 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:29.238407 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:29.242393 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:29.242443 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:29.242443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:29.242543 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:29.245582 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:29.246089 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:29.246095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:29.246096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:29.246262 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:29.246495 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:29.246495 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:29.246496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:29.247050 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:29.247237 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:29.247783 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 18:50:29.248172 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260811 18:50:29.263386 1986 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:29.264365 1986 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:29.271260 1986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Generated new TSK 0
I20260811 18:50:29.271638 1986 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:50:30.020609 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40671
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:30.132391 2052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:30.132709 2052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:30.132750 2052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:30.134634 2052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:30.134696 2052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:30.134727 2052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:50:30.136547 2052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.215.20:38867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:30.136818 2052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:30.137079 2052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:30.143603 2063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.143603 2060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.143703 2061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:30.144169 2052 server_base.cc:1034] running on GCE node
I20260811 18:50:30.144407 2052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:30.144739 2052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:30.145918 2052 hybrid_clock.cc:648] HybridClock initialized: now 1786474230145899 us; error 36 us; skew 500 ppm
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:30.149394 2052 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:50:30.150048 2052 webserver.cc:460] Webserver started at http://127.9.215.1:41801/ using document root <none> and password file <none>
I20260811 18:50:30.150360 2052 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:30.150447 2052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:30.150617 2052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:30.168975 2052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "621772e058f54b809779aa36a295d9eb"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "64a59ca33782d676c9dad0b7fcee1e7a"
server_key_iv: "e2099cf5edfa2574fb8babf4f9d77cc8"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.169489 2052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "621772e058f54b809779aa36a295d9eb"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "64a59ca33782d676c9dad0b7fcee1e7a"
server_key_iv: "e2099cf5edfa2574fb8babf4f9d77cc8"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.175961 2052 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260811 18:50:30.192103 2080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.192386 2052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:30.192489 2052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "621772e058f54b809779aa36a295d9eb"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "64a59ca33782d676c9dad0b7fcee1e7a"
server_key_iv: "e2099cf5edfa2574fb8babf4f9d77cc8"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.192598 2052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:30.208307 2052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:30.210409 2052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:30.210577 2052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:30.210811 2052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:30.211199 2052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:30.211239 2052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.211380 2052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:30.211491 2052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.218794 2052 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:44071
I20260811 18:50:30.218864 2334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:44071 every 8 connection(s)
I20260811 18:50:30.219269 2052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:30.228842 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 2052
I20260811 18:50:30.228958 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:30.229138 10076 external_mini_cluster.cc:1428] Setting key 4e8fb6891da8fc5ce3f0fa9dd6c43450
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:30.235334 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.220187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:45587 (local address 127.9.215.62:40671)
0811 18:50:30.223639 (+ 3452us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:30.223641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:30.223910 (+ 269us) server_negotiation.cc:407] Connection header received
0811 18:50:30.224474 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:30.224476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:30.224491 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:30.224564 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:30.225216 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.227667 (+ 2451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.229418 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.229732 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.229816 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.232298 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:30.232300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:30.232300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:30.232310 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:30.233946 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:30.234356 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:30.234357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:30.234358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:30.234420 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:30.234660 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:30.234661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:30.234661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:30.234783 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:30.234840 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:30.235130 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 18:50:30.235232 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3412,"thread_start_us":80,"threads_started":1}
I20260811 18:50:30.235873 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.220176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40671 (local address 127.9.215.1:45587)
0811 18:50:30.223747 (+ 3571us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:30.223799 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:30.224367 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:30.224643 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:30.224647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:30.224871 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:30.225105 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.225111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.227831 (+ 2720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.227833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.229292 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.229296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.229856 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.229857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.230019 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.230557 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:30.230566 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:30.232181 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:30.234034 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:30.234042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:30.234045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:30.234281 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:30.234480 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:30.234481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:30.234481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:30.234600 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:30.234840 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:30.234846 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:30.235003 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 18:50:30.235098 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:50:30.235264 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3520,"thread_start_us":92,"threads_started":1}
I20260811 18:50:30.236382 2336 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40671
I20260811 18:50:30.236483 2336 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:30.236704 2336 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:30.237242 1896 ts_manager.cc:194] Registered new tserver with Master: 621772e058f54b809779aa36a295d9eb (127.9.215.1:44071)
I20260811 18:50:30.238023 1896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:45587
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:50:30.266381 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40671
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:30.367873 2345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:30.368120 2345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:30.368158 2345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:30.369513 2345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:30.369572 2345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:30.369613 2345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:50:30.371665 2345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.215.20:38867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:30.371974 2345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:30.372241 2345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:30.377750 2356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.377979 2355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.377864 2359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:30.378481 2345 server_base.cc:1034] running on GCE node
I20260811 18:50:30.378707 2345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:30.378974 2345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:30.380126 2345 hybrid_clock.cc:648] HybridClock initialized: now 1786474230380105 us; error 44 us; skew 500 ppm
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:30.382851 2345 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:50:30.383332 2345 webserver.cc:460] Webserver started at http://127.9.215.2:34165/ using document root <none> and password file <none>
I20260811 18:50:30.383715 2345 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:30.383780 2345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:30.383945 2345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:30.404392 2345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "7f5e21207bae42b085e55eb62d1d3c1a"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "9313edbd774a4cebe4bbe166379bb37b"
server_key_iv: "e361652efa9f1bdb77196d940b3a8c1b"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.404946 2345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "7f5e21207bae42b085e55eb62d1d3c1a"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "9313edbd774a4cebe4bbe166379bb37b"
server_key_iv: "e361652efa9f1bdb77196d940b3a8c1b"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.408722 2345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:50:30.425946 2378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.431902 2345 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260811 18:50:30.432045 2345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "7f5e21207bae42b085e55eb62d1d3c1a"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "9313edbd774a4cebe4bbe166379bb37b"
server_key_iv: "e361652efa9f1bdb77196d940b3a8c1b"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.432174 2345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:30.455963 2345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:30.458843 2345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:30.459017 2345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:30.459280 2345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:30.459739 2345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:30.459793 2345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.459824 2345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:30.459841 2345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.467288 2345 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42097
I20260811 18:50:30.468052 2345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:30.469836 2598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42097 every 8 connection(s)
I20260811 18:50:30.475931 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 2345
I20260811 18:50:30.476052 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:30.476257 10076 external_mini_cluster.cc:1428] Setting key b939c7975d6066c1ce91cb4c1db19951
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:30.486415 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.471661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:53305 (local address 127.9.215.62:40671)
0811 18:50:30.471727 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:30.471729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:30.472052 (+ 323us) server_negotiation.cc:407] Connection header received
0811 18:50:30.472799 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:30.472800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:30.472813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:30.472843 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:30.473749 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.474487 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.479385 (+ 4898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.479740 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.479807 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.482826 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:30.482829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:30.482829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:30.482838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:30.484585 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:30.485164 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:30.485166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:30.485166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:30.485248 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:30.485564 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:30.485565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:30.485566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:30.485717 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:30.485763 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:30.486215 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 18:50:30.486314 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:50:30.487329 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.471284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40671 (local address 127.9.215.2:53305)
0811 18:50:30.471906 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:30.471957 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:30.472661 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:30.473044 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:30.473048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:30.473337 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:30.473627 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.473634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.474609 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.474609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.479259 (+ 4650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.479264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.480025 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.480026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.480272 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.480938 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:30.480949 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:30.482680 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:30.484700 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:30.484710 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:30.484714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:30.485059 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:30.485324 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:30.485325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:30.485325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:30.485483 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:30.485822 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:30.485834 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:30.486043 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 18:50:30.486842 (+ 799us) client_negotiation.cc:240] Negotiation successful
0811 18:50:30.487035 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":117,"threads_started":1}
I20260811 18:50:30.488073 2599 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40671
I20260811 18:50:30.488173 2599 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:30.488382 2599 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:30.488729 1896 ts_manager.cc:194] Registered new tserver with Master: 7f5e21207bae42b085e55eb62d1d3c1a (127.9.215.2:42097)
I20260811 18:50:30.489220 1896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:53305
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:50:30.523362 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40671
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:30.648401 2608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:30.648949 2608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:30.649089 2608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:30.651250 2608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:30.651408 2608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:30.651538 2608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:50:30.654485 2608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.215.20:38867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:30.654840 2608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:30.655221 2608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:30.662165 2618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.662665 2619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.662931 2624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:30.664567 2608 server_base.cc:1034] running on GCE node
I20260811 18:50:30.664849 2608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:30.665205 2608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:30.666395 2608 hybrid_clock.cc:648] HybridClock initialized: now 1786474230666371 us; error 41 us; skew 500 ppm
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:30.669862 2608 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:50:30.670523 2608 webserver.cc:460] Webserver started at http://127.9.215.3:33013/ using document root <none> and password file <none>
I20260811 18:50:30.670785 2608 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:30.670828 2608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:30.670931 2608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:30.686836 2608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "f80cc4767d8e4b48863d35fe04e39f87"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "45e95f9a66e88c97a44ae4b760a5bc13"
server_key_iv: "fada22031154a75d38adbb35f6caeaf4"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.687374 2608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "f80cc4767d8e4b48863d35fe04e39f87"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "45e95f9a66e88c97a44ae4b760a5bc13"
server_key_iv: "fada22031154a75d38adbb35f6caeaf4"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.691979 2608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260811 18:50:30.707655 2644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.707855 2608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:50:30.707957 2608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "f80cc4767d8e4b48863d35fe04e39f87"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "45e95f9a66e88c97a44ae4b760a5bc13"
server_key_iv: "fada22031154a75d38adbb35f6caeaf4"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:30.708060 2608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:30.745865 2608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:30.748713 2608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:30.748883 2608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:30.749181 2608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:30.749714 2608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:30.749768 2608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.749797 2608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:30.749861 2608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:30.758335 2608 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44561
I20260811 18:50:30.758374 2891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44561 every 8 connection(s)
I20260811 18:50:30.758927 2608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:30.764343 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 2608
I20260811 18:50:30.764468 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:30.764648 10076 external_mini_cluster.cc:1428] Setting key 6fc375b04cc2a6bd8e60ce9d4a8f9639
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:30.773150 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.760726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:37291 (local address 127.9.215.62:40671)
0811 18:50:30.760777 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:30.760779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:30.761019 (+ 240us) server_negotiation.cc:407] Connection header received
0811 18:50:30.761802 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:30.761804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:30.761817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:30.761847 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:30.762959 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.763756 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.766085 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.766401 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.766446 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.769484 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:30.769486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:30.769486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:30.769495 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:30.771243 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:30.771888 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:30.771890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:30.771891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:30.771959 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:30.772278 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:30.772279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:30.772280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:30.772439 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:30.772486 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:30.772954 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 18:50:30.773051 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:50:30.774147 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:30.760316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40671 (local address 127.9.215.3:37291)
0811 18:50:30.760863 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:30.760916 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:30.761659 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:30.762052 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:30.762057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:30.762379 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:30.762830 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.762838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.763891 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.763893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.765977 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:30.765981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:30.766566 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:30.766567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:30.766739 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:30.767427 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:30.767441 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:30.769342 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:30.771364 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:30.771375 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:30.771378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:30.771785 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:30.772034 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:30.772035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:30.772035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:30.772200 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:30.772539 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:30.772547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:30.772781 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 18:50:30.773662 (+ 881us) client_negotiation.cc:240] Negotiation successful
0811 18:50:30.773883 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":169,"threads_started":1}
I20260811 18:50:30.774960 2892 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40671
I20260811 18:50:30.775067 2892 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:30.775396 2892 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:30.776149 1896 ts_manager.cc:194] Registered new tserver with Master: f80cc4767d8e4b48863d35fe04e39f87 (127.9.215.3:44561)
I20260811 18:50:30.776701 1896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:37291
I20260811 18:50:30.778048 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:30.778781 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 2052
I20260811 18:50:30.819015 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 2345
I20260811 18:50:30.833427 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 2608
I20260811 18:50:30.846592 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 1829
I20260811 18:50:30.871534 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40671
--webserver_interface=127.9.215.62
--webserver_port=46029
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:40671
--ranger_config_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:30.975390 2896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:30.975679 2896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:30.975706 2896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:30.976953 2896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:30.977002 2896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:30.977015 2896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:30.977026 2896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:30.977037 2896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:30.978832 2896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:40671
--ranger_config_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=46029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:30.979100 2896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:30.979545 2896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:30.984746 2915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.984751 2912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:30.984746 2913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:30.985339 2896 server_base.cc:1034] running on GCE node
I20260811 18:50:30.985625 2896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:30.986008 2896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:30.987185 2896 hybrid_clock.cc:648] HybridClock initialized: now 1786474230987162 us; error 38 us; skew 500 ppm
Aug 11 18:50:30 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:30.990361 2896 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:30.990950 2896 webserver.cc:460] Webserver started at http://127.9.215.62:46029/ using document root <none> and password file <none>
I20260811 18:50:30.991209 2896 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:30.991281 2896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474230, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:31.062247 2896 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:50:31.063225 2931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.063436 2896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:50:31.063601 2896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "a02b960edd824c97bdd95f4ee59245a5"
format_stamp: "Formatted at 2026-08-11 18:50:29 on dist-test-slave-pww0"
server_key: "16523a9e811cddec4cd7019fce444f7c"
server_key_iv: "672e13ab5e21355c3d6b9ed6f6385479"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:31.064100 2896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:31.088506 2896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:31.090288 2896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:31.090423 2896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:31.094774 2896 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:40671
I20260811 18:50:31.094825 3033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:40671 every 8 connection(s)
I20260811 18:50:31.095237 2896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:31.096709 3034 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:50:31.097383 3034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Bootstrap starting.
I20260811 18:50:31.100333 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 2896
I20260811 18:50:31.100700 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:44071
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:40671
--webserver_port=41801
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:31.106092 3034 log.cc:824] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:31.108052 3034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:31.108325 3034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Bootstrap complete.
I20260811 18:50:31.110642 3034 raft_consensus.cc:348] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:31.110898 3034 raft_consensus.cc:729] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02b960edd824c97bdd95f4ee59245a5, State: Initialized, Role: FOLLOWER
I20260811 18:50:31.111057 3034 consensus_queue.cc:260] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:31.111133 3034 raft_consensus.cc:388] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:31.111186 3034 raft_consensus.cc:482] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:31.111234 3034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:50:31.112409 3034 raft_consensus.cc:504] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:31.112516 3034 leader_election.cc:302] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a02b960edd824c97bdd95f4ee59245a5; no voters:
I20260811 18:50:31.112735 3034 leader_election.cc:288] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:50:31.112816 3041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:50:31.113009 3034 sys_catalog.cc:564] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:31.113039 3041 raft_consensus.cc:686] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [term 2 LEADER]: Becoming Leader. State: Replica: a02b960edd824c97bdd95f4ee59245a5, State: Running, Role: LEADER
I20260811 18:50:31.113155 3041 consensus_queue.cc:237] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } }
I20260811 18:50:31.113516 3041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a02b960edd824c97bdd95f4ee59245a5. Latest consensus state: current_term: 2 leader_uuid: "a02b960edd824c97bdd95f4ee59245a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } } }
I20260811 18:50:31.113615 3041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:31.113889 3042 sys_catalog.cc:455] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a02b960edd824c97bdd95f4ee59245a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a02b960edd824c97bdd95f4ee59245a5" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40671 } } }
I20260811 18:50:31.114018 3042 sys_catalog.cc:458] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:31.114311 3050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:31.114862 3050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:31.115054 3050 catalog_manager.cc:1259] Loaded cluster ID: 679764d61aa04d438d3890158463b295
I20260811 18:50:31.115139 3050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:31.115221 3034 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:50:31.118777 3050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:31.119513 3050 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Loaded TSK: 0
I20260811 18:50:31.119931 3050 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:50:31.199407 3038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:31.199843 3038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:31.199945 3038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:31.202009 3038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:31.202189 3038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:31.202287 3038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:50:31.204996 3038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:44071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=41801
--tserver_master_addrs=127.9.215.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:31.205341 3038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:31.205698 3038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:31.211725 3102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:31.212071 3096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:31.212252 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:31.213204 3038 server_base.cc:1034] running on GCE node
I20260811 18:50:31.213464 3038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:31.213742 3038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:31.217018 3038 hybrid_clock.cc:648] HybridClock initialized: now 1786474231216993 us; error 34 us; skew 500 ppm
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:31.220022 3038 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:50:31.220636 3038 webserver.cc:460] Webserver started at http://127.9.215.1:41801/ using document root <none> and password file <none>
I20260811 18:50:31.220935 3038 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:31.221016 3038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:31.238368 3038 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:31.239655 3122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.239969 3038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:31.240087 3038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "621772e058f54b809779aa36a295d9eb"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "64a59ca33782d676c9dad0b7fcee1e7a"
server_key_iv: "e2099cf5edfa2574fb8babf4f9d77cc8"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:31.240592 3038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:31.258862 3038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:31.261274 3038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:31.261507 3038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:31.261775 3038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:31.262157 3038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:31.262198 3038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.262229 3038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:31.262269 3038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.269461 3038 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:44071
I20260811 18:50:31.270169 3038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:31.272166 3332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:44071 every 8 connection(s)
I20260811 18:50:31.273984 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 3038
I20260811 18:50:31.274329 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:42097
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:40671
--webserver_port=34165
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:31.285789 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.273020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40671 (local address 127.9.215.1:46427)
0811 18:50:31.275604 (+ 2584us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:31.275671 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:31.276369 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:31.277025 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:31.277029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:31.277256 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:31.277462 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.277467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.278629 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.278631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.279507 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.279510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.280110 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.280111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.280260 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.280800 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:31.280809 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:31.282122 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:31.284015 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:31.284023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:31.284025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:31.284295 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:31.284538 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:31.284538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:31.284539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:31.284653 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:31.285098 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:31.285105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:31.285288 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 18:50:31.285388 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 18:50:31.285552 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2525,"thread_start_us":103,"threads_started":1}
I20260811 18:50:31.286410 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.273123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:46427 (local address 127.9.215.62:40671)
0811 18:50:31.276606 (+ 3483us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.276615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.276645 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:50:31.276782 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.276786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.276833 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.276953 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.277602 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.278518 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.279583 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.280010 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.280111 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.282255 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:31.282259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:31.282260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:31.282272 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:31.283932 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.284382 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.284385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.284385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.284478 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.284711 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.284712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.284712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.284986 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:31.285069 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:31.286012 (+ 943us) server_negotiation.cc:299] Negotiation successful
0811 18:50:31.286147 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3421,"thread_start_us":102,"threads_started":1}
I20260811 18:50:31.287554 3333 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40671
I20260811 18:50:31.287678 3333 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.287915 3333 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
W20260811 18:50:31.288928 3333 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:40671 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:50:31.289006 3333 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.289084 3333 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:31.289484 3333 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.289562 3333 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
W20260811 18:50:31.289887 3333 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:50:31.385524 3343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:31.385836 3343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:31.385880 3343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:31.387815 3343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:31.387883 3343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:31.387925 3343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:50:31.390350 3343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:42097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=34165
--tserver_master_addrs=127.9.215.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:31.390761 3343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:31.391068 3343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:31.397615 3361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:31.398051 3356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:31.399113 3353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:31.399449 3343 server_base.cc:1034] running on GCE node
I20260811 18:50:31.399710 3343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:31.400007 3343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:31.401168 3343 hybrid_clock.cc:648] HybridClock initialized: now 1786474231401142 us; error 39 us; skew 500 ppm
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:31.404397 3343 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:50:31.404928 3343 webserver.cc:460] Webserver started at http://127.9.215.2:34165/ using document root <none> and password file <none>
I20260811 18:50:31.405143 3343 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:31.405202 3343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:31.423018 3343 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:31.424078 3376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.424315 3343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:31.424466 3343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "7f5e21207bae42b085e55eb62d1d3c1a"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "9313edbd774a4cebe4bbe166379bb37b"
server_key_iv: "e361652efa9f1bdb77196d940b3a8c1b"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:31.425052 3343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:31.438199 3343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:31.440385 3343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:31.440536 3343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:31.440747 3343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:31.441129 3343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:31.441234 3343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.441315 3343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:31.441376 3343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.450340 3343 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42097
I20260811 18:50:31.450443 3599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42097 every 8 connection(s)
I20260811 18:50:31.451014 3343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:31.453218 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 3343
I20260811 18:50:31.453547 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.215.20:38867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:44561
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:40671
--webserver_port=33013
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:31.462459 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.451890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:48647 (local address 127.9.215.62:40671)
0811 18:50:31.451966 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.451969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.452211 (+ 242us) server_negotiation.cc:407] Connection header received
0811 18:50:31.452946 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.452948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.452963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.453052 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.453901 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.454525 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.455705 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.455970 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.456017 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.458847 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:31.458849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:31.458849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:31.458858 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:31.460602 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.461192 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.461194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.461195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.461262 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.461588 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.461589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.461590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.461742 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:31.461790 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:31.462250 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 18:50:31.462354 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:50:31.463397 3606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.451829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40671 (local address 127.9.215.2:48647)
0811 18:50:31.452049 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:31.452109 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:31.452779 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:31.453214 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:31.453220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:31.453476 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:31.453772 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.453780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.454626 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.454626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.455596 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.455600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.456136 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.456137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.456301 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.456947 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:31.456959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:31.458708 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:31.460715 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:31.460729 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:31.460732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:31.461082 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:31.461341 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:31.461342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:31.461343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:31.461507 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:31.461838 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:31.461847 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:31.462066 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 18:50:31.462857 (+ 791us) client_negotiation.cc:240] Negotiation successful
0811 18:50:31.463063 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260811 18:50:31.464295 3600 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40671
I20260811 18:50:31.464427 3600 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.464648 3600 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
W20260811 18:50:31.465479 3600 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:40671 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:50:31.465561 3600 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.465636 3600 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:31.466041 3600 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.466132 3600 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
W20260811 18:50:31.466464 3600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:50:31.549641 3608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:31.549954 3608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:31.549991 3608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:31.551561 3608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:31.551623 3608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:31.551656 3608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:50:31.553390 3608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.215.20:38867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:44561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=33013
--tserver_master_addrs=127.9.215.62:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:31.553663 3608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:31.553920 3608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:31.559365 3618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:31.559362 3623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:31.559355 3619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:31.559708 3608 server_base.cc:1034] running on GCE node
I20260811 18:50:31.559998 3608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:31.560279 3608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:31.561465 3608 hybrid_clock.cc:648] HybridClock initialized: now 1786474231561447 us; error 29 us; skew 500 ppm
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:31.565209 3608 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:50:31.565887 3608 webserver.cc:460] Webserver started at http://127.9.215.3:33013/ using document root <none> and password file <none>
I20260811 18:50:31.566203 3608 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:31.566296 3608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:31.592967 3608 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:31.594017 3636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.594269 3608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:31.594391 3608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "f80cc4767d8e4b48863d35fe04e39f87"
format_stamp: "Formatted at 2026-08-11 18:50:30 on dist-test-slave-pww0"
server_key: "45e95f9a66e88c97a44ae4b760a5bc13"
server_key_iv: "fada22031154a75d38adbb35f6caeaf4"
server_key_version: "kuduclusterkey@0"
I20260811 18:50:31.594857 3608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:31.616101 3608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:31.618208 3608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:31.618458 3608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:31.618820 3608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:31.619247 3608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:31.619313 3608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.619354 3608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:31.619381 3608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:31.626714 3608 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44561
I20260811 18:50:31.627231 3608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:31.627405 3843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44561 every 8 connection(s)
I20260811 18:50:31.628314 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 3608
Aug 11 18:50:31 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:31.641103 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.628379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:52921 (local address 127.9.215.62:40671)
0811 18:50:31.628431 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.628433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.628704 (+ 271us) server_negotiation.cc:407] Connection header received
0811 18:50:31.629420 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.629421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.629433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.629517 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.630899 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.631661 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.633206 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.633547 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.633633 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.636206 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:31.636207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:31.636208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:31.636216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:31.638853 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.639305 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.639306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.639306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.639366 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.639714 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.639715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.639715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.640110 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:31.640161 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:31.640882 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 18:50:31.640999 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:50:31.641427 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.628385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40671 (local address 127.9.215.3:52921)
0811 18:50:31.628545 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:31.628599 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:31.629300 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:31.629665 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:31.629670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:31.629924 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:31.630789 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.630797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.631767 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.631769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.633107 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:31.633110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.633670 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.633672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:31.633820 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.634417 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:31.634426 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:31.636081 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:31.638958 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:31.638968 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:31.638971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:31.639229 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:31.639424 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:31.639425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:31.639425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:31.639623 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:31.640224 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:31.640232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:31.640703 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 18:50:31.640998 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 18:50:31.641176 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260811 18:50:31.642030 3844 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40671
I20260811 18:50:31.642172 3844 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.642407 3844 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
W20260811 18:50:31.643213 3844 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:40671 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:50:31.643287 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:31.628894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.9.215.62:40671)
0811 18:50:31.629339 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:31.629341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:31.629512 (+ 171us) server_negotiation.cc:407] Connection header received
0811 18:50:31.629545 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:31.629546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:31.629559 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:31.629633 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:31.631445 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.632361 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.634391 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:31.634706 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:31.634777 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:31.639517 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:31.639518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:31.639518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:31.639526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:31.641894 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.642289 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.642290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.642290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.642351 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:31.642531 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:31.642532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:31.642532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:31.642658 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:31.642824 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:31.643101 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 18:50:31.643197 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":65,"threads_started":1}
I20260811 18:50:31.643301 3844 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.643371 3844 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:31.643816 3844 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:31.643908 3844 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
W20260811 18:50:31.644263 3844 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 18:50:32 dist-test-slave-pww0 krb5kdc[933](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:32.290084 3333 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:32.290287 3333 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:32.290726 2955 ts_manager.cc:194] Registered new tserver with Master: 621772e058f54b809779aa36a295d9eb (127.9.215.1:44071)
I20260811 18:50:32.291407 2955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:46427
I20260811 18:50:32.466679 3600 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:32.466862 3600 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:32.467303 2955 ts_manager.cc:194] Registered new tserver with Master: 7f5e21207bae42b085e55eb62d1d3c1a (127.9.215.2:42097)
I20260811 18:50:32.467854 2955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:48647
I20260811 18:50:32.644455 3844 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:32.644624 3844 heartbeater.cc:511] Master 127.9.215.62:40671 requested a full tablet report, sending...
I20260811 18:50:32.645006 2955 ts_manager.cc:194] Registered new tserver with Master: f80cc4767d8e4b48863d35fe04e39f87 (127.9.215.3:44561)
I20260811 18:50:32.645557 2955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:52921
I20260811 18:50:32.655109 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:32.664641 3888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.656203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.9.215.62:40671)
0811 18:50:32.656335 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.656337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.656390 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:50:32.656458 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.656459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.656471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.656549 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:32.657268 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.658082 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.659371 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.659749 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.659829 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.661446 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:32.661448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:32.661448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:32.661458 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:32.663228 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:32.663713 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:32.663715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:32.663715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:32.663790 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:32.664001 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:32.664002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:32.664003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:32.664168 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:32.664278 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.664429 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.664522 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260811 18:50:32.667201 2955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:32.669577 2955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:32.680613 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.674473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:44071 (local address 127.0.0.1:51476)
0811 18:50:32.675573 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.675632 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.675723 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.676051 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.676055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.676058 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:32.676152 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.676155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.677743 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.677744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.679152 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.679155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.680285 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.680286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.680382 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.680398 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:50:32.680464 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:50:32.680522 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"thread_start_us":132,"threads_started":1}
I20260811 18:50:32.680944 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.675099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.9.215.2:42097)
0811 18:50:32.675265 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.675270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.676339 (+ 1069us) server_negotiation.cc:407] Connection header received
0811 18:50:32.676453 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.676456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.676474 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.676530 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:32.676782 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.677999 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.679266 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.680427 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.680492 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.680564 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.680810 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.680857 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260811 18:50:32.681300 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.674899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.9.215.1:44071)
0811 18:50:32.675056 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.675062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.675817 (+ 755us) server_negotiation.cc:407] Connection header received
0811 18:50:32.675860 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.675863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.675880 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.675954 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:32.676242 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.677656 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.679254 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.680086 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.680165 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.680245 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.681166 (+ 921us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.681218 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260811 18:50:32.681736 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.674476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42097 (local address 127.0.0.1:40302)
0811 18:50:32.676271 (+ 1795us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.676281 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.676370 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.676647 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.676648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.676650 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:32.676705 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.676707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.678123 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.678124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.679157 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.679161 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.680632 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.680633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.680712 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.680716 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:50:32.681304 (+ 588us) client_negotiation.cc:240] Negotiation successful
0811 18:50:32.681361 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1777,"thread_start_us":130,"threads_started":1}
I20260811 18:50:32.682147 3219 tablet_service.cc:1467] Processing CreateTablet for tablet bbbba3533696434c82c67a95cc42163b (DEFAULT_TABLE table=test-table [id=43bbc274f090460493f190a57abd95ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:32.682554 3219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbba3533696434c82c67a95cc42163b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:32.682718 3470 tablet_service.cc:1467] Processing CreateTablet for tablet bbbba3533696434c82c67a95cc42163b (DEFAULT_TABLE table=test-table [id=43bbc274f090460493f190a57abd95ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:32.683082 3470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbba3533696434c82c67a95cc42163b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:32.687409 3902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.674895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.9.215.3:44561)
0811 18:50:32.683587 (+ 8692us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.683593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.683606 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:32.683642 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.683644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.683662 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.683754 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:32.683976 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.685007 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.686340 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.686953 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.687030 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.687108 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.687291 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.687331 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8645,"thread_start_us":64,"threads_started":1}
I20260811 18:50:32.687400 3900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.674825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:44561 (local address 127.0.0.1:35918)
0811 18:50:32.681051 (+ 6226us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.681061 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.681093 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.683854 (+ 2761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.683855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.683857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:32.683900 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.683901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.685113 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.685114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.686181 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.686185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.687150 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.687151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.687216 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.687220 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:50:32.687272 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 18:50:32.687322 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6212,"thread_start_us":81,"threads_started":1}
I20260811 18:50:32.688268 3720 tablet_service.cc:1467] Processing CreateTablet for tablet bbbba3533696434c82c67a95cc42163b (DEFAULT_TABLE table=test-table [id=43bbc274f090460493f190a57abd95ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:32.688386 3908 tablet_bootstrap.cc:492] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Bootstrap starting.
I20260811 18:50:32.688634 3720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbba3533696434c82c67a95cc42163b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:32.689124 3908 tablet_bootstrap.cc:654] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:32.689443 3908 log.cc:824] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:32.690241 3908 tablet_bootstrap.cc:492] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: No bootstrap required, opened a new log
I20260811 18:50:32.690320 3908 ts_tablet_manager.cc:1397] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:50:32.693595 3908 raft_consensus.cc:348] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.693814 3908 raft_consensus.cc:374] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:32.693859 3908 raft_consensus.cc:729] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5e21207bae42b085e55eb62d1d3c1a, State: Initialized, Role: FOLLOWER
I20260811 18:50:32.694046 3908 consensus_queue.cc:260] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.694392 3908 ts_tablet_manager.cc:1428] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:32.694422 3600 heartbeater.cc:503] Master 127.9.215.62:40671 was elected leader, sending a full tablet report...
I20260811 18:50:32.698243 3906 tablet_bootstrap.cc:492] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: Bootstrap starting.
I20260811 18:50:32.698490 3913 tablet_bootstrap.cc:492] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Bootstrap starting.
I20260811 18:50:32.699301 3913 tablet_bootstrap.cc:654] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:32.699406 3906 tablet_bootstrap.cc:654] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:32.699682 3913 log.cc:824] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:32.699887 3906 log.cc:824] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: Log is configured to *not* fsync() on all Append() calls
W20260811 18:50:32.702265 3603 tablet.cc:2369] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:32.702937 3913 tablet_bootstrap.cc:492] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: No bootstrap required, opened a new log
I20260811 18:50:32.703039 3913 ts_tablet_manager.cc:1397] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 18:50:32.705801 3913 raft_consensus.cc:348] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.706101 3913 raft_consensus.cc:374] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:32.706179 3913 raft_consensus.cc:729] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80cc4767d8e4b48863d35fe04e39f87, State: Initialized, Role: FOLLOWER
I20260811 18:50:32.706410 3913 consensus_queue.cc:260] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.706743 3913 ts_tablet_manager.cc:1428] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260811 18:50:32.706792 3844 heartbeater.cc:503] Master 127.9.215.62:40671 was elected leader, sending a full tablet report...
I20260811 18:50:32.708112 3906 tablet_bootstrap.cc:492] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: No bootstrap required, opened a new log
I20260811 18:50:32.708225 3906 ts_tablet_manager.cc:1397] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260811 18:50:32.710773 3906 raft_consensus.cc:348] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.711009 3906 raft_consensus.cc:374] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:32.711614 3906 raft_consensus.cc:729] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621772e058f54b809779aa36a295d9eb, State: Initialized, Role: FOLLOWER
I20260811 18:50:32.711884 3906 consensus_queue.cc:260] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.712255 3906 ts_tablet_manager.cc:1428] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 18:50:32.712280 3333 heartbeater.cc:503] Master 127.9.215.62:40671 was elected leader, sending a full tablet report...
Aug 11 18:50:32 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
W20260811 18:50:32.774402 3338 tablet.cc:2369] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:50:32.880605 3845 tablet.cc:2369] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:32.914017 3917 raft_consensus.cc:482] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:32.914254 3917 raft_consensus.cc:504] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.914716 3917 leader_election.cc:288] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 621772e058f54b809779aa36a295d9eb (127.9.215.1:44071), f80cc4767d8e4b48863d35fe04e39f87 (127.9.215.3:44561)
I20260811 18:50:32.919014 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.915053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:47901 (local address 127.9.215.1:44071)
0811 18:50:32.915104 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.915106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.915270 (+ 164us) server_negotiation.cc:407] Connection header received
0811 18:50:32.915384 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.915385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.915399 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.915428 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:32.915756 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.916456 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.917969 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.918579 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.918638 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.918651 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.918886 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.918935 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:50:32.919763 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.914971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:44071 (local address 127.9.215.2:47901)
0811 18:50:32.915181 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.915194 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.915302 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.915544 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.915545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.915548 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:32.915667 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.915669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.916561 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.916562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.917840 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.917844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.918713 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.918714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.918804 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.918808 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:50:32.919510 (+ 702us) client_negotiation.cc:240] Negotiation successful
0811 18:50:32.919600 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":48,"thread_start_us":122,"threads_started":1}
I20260811 18:50:32.920322 3902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.915433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:54079 (local address 127.9.215.3:44561)
0811 18:50:32.915503 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.915505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.915646 (+ 141us) server_negotiation.cc:407] Connection header received
0811 18:50:32.915773 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.915775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.915790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.915821 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:32.916125 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.916753 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.918080 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.918636 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.918684 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.918694 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.920153 (+ 1459us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.920246 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:50:32.920418 3256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbba3533696434c82c67a95cc42163b" candidate_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621772e058f54b809779aa36a295d9eb" is_pre_election: true
I20260811 18:50:32.920609 3256 raft_consensus.cc:2454] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5e21207bae42b085e55eb62d1d3c1a in term 0.
I20260811 18:50:32.920804 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.915227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:44561 (local address 127.9.215.2:54079)
0811 18:50:32.915574 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:32.915584 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:32.915690 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:32.915894 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:32.915895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:32.915896 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:32.916025 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.916028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.916852 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.916853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.917968 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:32.917971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.918772 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.918773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:32.918852 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.918856 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:50:32.920652 (+ 1796us) client_negotiation.cc:240] Negotiation successful
0811 18:50:32.920704 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":76,"threads_started":1}
I20260811 18:50:32.920926 3395 leader_election.cc:302] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 621772e058f54b809779aa36a295d9eb, 7f5e21207bae42b085e55eb62d1d3c1a; no voters:
I20260811 18:50:32.921096 3917 raft_consensus.cc:2781] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:32.921171 3917 raft_consensus.cc:482] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:32.921195 3917 raft_consensus.cc:3037] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:32.921294 3759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbba3533696434c82c67a95cc42163b" candidate_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80cc4767d8e4b48863d35fe04e39f87" is_pre_election: true
I20260811 18:50:32.921456 3759 raft_consensus.cc:2454] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5e21207bae42b085e55eb62d1d3c1a in term 0.
I20260811 18:50:32.922137 3917 raft_consensus.cc:504] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.922269 3917 leader_election.cc:288] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [CANDIDATE]: Term 1 election: Requested vote from peers 621772e058f54b809779aa36a295d9eb (127.9.215.1:44071), f80cc4767d8e4b48863d35fe04e39f87 (127.9.215.3:44561)
I20260811 18:50:32.922732 3759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbba3533696434c82c67a95cc42163b" candidate_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80cc4767d8e4b48863d35fe04e39f87"
I20260811 18:50:32.922822 3759 raft_consensus.cc:3037] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:32.923043 3256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbba3533696434c82c67a95cc42163b" candidate_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621772e058f54b809779aa36a295d9eb"
I20260811 18:50:32.923143 3256 raft_consensus.cc:3037] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:32.923763 3759 raft_consensus.cc:2454] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5e21207bae42b085e55eb62d1d3c1a in term 1.
I20260811 18:50:32.923988 3256 raft_consensus.cc:2454] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5e21207bae42b085e55eb62d1d3c1a in term 1.
I20260811 18:50:32.923981 3395 leader_election.cc:302] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f5e21207bae42b085e55eb62d1d3c1a, f80cc4767d8e4b48863d35fe04e39f87; no voters:
I20260811 18:50:32.924065 3917 raft_consensus.cc:2781] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:32.924237 3917 raft_consensus.cc:686] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 1 LEADER]: Becoming Leader. State: Replica: 7f5e21207bae42b085e55eb62d1d3c1a, State: Running, Role: LEADER
I20260811 18:50:32.924340 3917 consensus_queue.cc:237] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } }
I20260811 18:50:32.925191 2955 catalog_manager.cc:5495] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f5e21207bae42b085e55eb62d1d3c1a (127.9.215.2). New cstate: current_term: 1 leader_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f5e21207bae42b085e55eb62d1d3c1a" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:32.962950 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:32.960015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.9.215.2:42097)
0811 18:50:32.960064 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:32.960066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:32.960082 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:32.960125 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:32.960126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:32.960165 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:32.960236 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:32.960382 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.961058 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.962005 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:32.962308 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:32.962381 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:32.962611 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:32.962725 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:32.962803 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:32.962843 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 18:50:32.962868 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:50:32.965279 3759 raft_consensus.cc:1261] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 1 FOLLOWER]: Refusing update from remote peer 7f5e21207bae42b085e55eb62d1d3c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:32.965664 3256 raft_consensus.cc:1261] T bbbba3533696434c82c67a95cc42163b P 621772e058f54b809779aa36a295d9eb [term 1 FOLLOWER]: Refusing update from remote peer 7f5e21207bae42b085e55eb62d1d3c1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:32.965840 3917 consensus_queue.cc:1035] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80cc4767d8e4b48863d35fe04e39f87" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:32.966126 3917 consensus_queue.cc:1035] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "621772e058f54b809779aa36a295d9eb" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 44071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:32.970808 3943 mvcc.cc:205] Tried to move back new op lower bound from 7317398458222718976 to 7317398458058596352. Current Snapshot: MvccSnapshot[applied={T|T < 7317398458222718976}]
I20260811 18:50:32.973735 2955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:32.973858 2955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:32.974826 2955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a02b960edd824c97bdd95f4ee59245a5: Sending DeleteTablet for 3 replicas of tablet bbbba3533696434c82c67a95cc42163b
I20260811 18:50:32.975414 3720 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbba3533696434c82c67a95cc42163b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:32 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:35918
I20260811 18:50:32.976135 3470 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbba3533696434c82c67a95cc42163b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:32 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:40302
I20260811 18:50:32.976596 3956 tablet_replica.cc:330] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: stopping tablet replica
I20260811 18:50:32.976828 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 3038
I20260811 18:50:32.976763 3956 raft_consensus.cc:2229] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:32.981050 3956 raft_consensus.cc:2258] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:32.981389 3954 tablet_replica.cc:330] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: stopping tablet replica
I20260811 18:50:32.981585 3954 raft_consensus.cc:2229] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:32.981675 3954 raft_consensus.cc:2258] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:33.064805 3956 ts_tablet_manager.cc:1905] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:33.066712 3956 ts_tablet_manager.cc:1918] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:33.066809 3956 log.cc:1196] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/bbbba3533696434c82c67a95cc42163b
I20260811 18:50:33.067066 3956 ts_tablet_manager.cc:1939] T bbbba3533696434c82c67a95cc42163b P 7f5e21207bae42b085e55eb62d1d3c1a: Deleting consensus metadata
I20260811 18:50:33.067615 2933 catalog_manager.cc:4841] TS 7f5e21207bae42b085e55eb62d1d3c1a (127.9.215.2:42097): tablet bbbba3533696434c82c67a95cc42163b (table test-table [id=43bbc274f090460493f190a57abd95ae]) successfully deleted
I20260811 18:50:33.069031 3954 ts_tablet_manager.cc:1905] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:33.070837 3954 ts_tablet_manager.cc:1918] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:33.070931 3954 log.cc:1196] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/bbbba3533696434c82c67a95cc42163b
I20260811 18:50:33.071231 3954 ts_tablet_manager.cc:1939] T bbbba3533696434c82c67a95cc42163b P f80cc4767d8e4b48863d35fe04e39f87: Deleting consensus metadata
I20260811 18:50:33.071758 2937 catalog_manager.cc:4841] TS f80cc4767d8e4b48863d35fe04e39f87 (127.9.215.3:44561): tablet bbbba3533696434c82c67a95cc42163b (table test-table [id=43bbc274f090460493f190a57abd95ae]) successfully deleted
W20260811 18:50:33.071934 2937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:44071 (error 108)
I20260811 18:50:33.072507 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:33.072368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:44071 (local address 127.0.0.1:51488)
0811 18:50:33.072425 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:33.072466 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:44071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:50:33.072649 2937 catalog_manager.cc:4584] TS 621772e058f54b809779aa36a295d9eb (127.9.215.1:44071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbbba3533696434c82c67a95cc42163b: Network error: Client connection negotiation failed: client connection to 127.9.215.1:44071: connect: Connection refused (error 111)
I20260811 18:50:33.079761 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 3343
I20260811 18:50:33.092944 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 3608
Aug 11 18:50:33 dist-test-slave-pww0 krb5kdc[933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for HTTP/127.9.215.20@KRBTEST.COM
I20260811 18:50:33.120471 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 2896
I20260811 18:50:33.155220 10076 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:50:33.510780 10076 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:50:33.510882 10076 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:50:33.857604 10076 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:50:33.857 UTC [990] LOG: received smart shutdown request
2026-08-11 18:50:33.862 UTC [990] LOG: background worker "logical replication launcher" (PID 1018) exited with exit code 1
2026-08-11 18:50:33.862 UTC [1000] LOG: shutting down
2026-08-11 18:50:34.052 UTC [990] LOG: database system is shut down
2026-08-11T18:50:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52339 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:34 dist-test-slave-pww0 krb5kdc[3982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:34 dist-test-slave-pww0 krb5kdc[3982](info): set up 2 sockets
Aug 11 18:50:34 dist-test-slave-pww0 krb5kdc[3982](info): commencing operation
krb5kdc: starting...
W20260811 18:50:35.374153 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:50:35.506551 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37263
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37263,127.9.215.61:44099,127.9.215.60:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:35.592579 4004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:35.592815 4004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:35.592849 4004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:35.594061 4004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:35.594106 4004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:35.594125 4004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:35.594141 4004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:35.594158 4004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:35.595819 4004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37263,127.9.215.61:44099,127.9.215.60:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:35.596088 4004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:35.596308 4004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:35.601369 4012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:35.601588 4013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:35.601702 4004 server_base.cc:1034] running on GCE node
W20260811 18:50:35.601364 4016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:35.602164 4004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:35.602536 4004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:35.603689 4004 hybrid_clock.cc:648] HybridClock initialized: now 1786474235603666 us; error 38 us; skew 500 ppm
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:35.606287 4004 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:35.606729 4004 webserver.cc:460] Webserver started at http://127.9.215.62:41885/ using document root <none> and password file <none>
I20260811 18:50:35.606918 4004 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:35.606957 4004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:35.607055 4004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:35.608183 4004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "90e2447e7e0f44888db849fdc3118d4f"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "54644c319f784df1ec2dca874d5241d2"
server_key_iv: "bbbf375f16968c1c664e4376096048e8"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.608502 4004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "90e2447e7e0f44888db849fdc3118d4f"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "54644c319f784df1ec2dca874d5241d2"
server_key_iv: "bbbf375f16968c1c664e4376096048e8"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.609750 4004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:35.610476 4029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:35.610682 4004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:35.610774 4004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "90e2447e7e0f44888db849fdc3118d4f"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "54644c319f784df1ec2dca874d5241d2"
server_key_iv: "bbbf375f16968c1c664e4376096048e8"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.610837 4004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:35.623724 4004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:35.625725 4004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:35.625852 4004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:35.629987 4004 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:37263
I20260811 18:50:35.630028 4136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:37263 every 8 connection(s)
I20260811 18:50:35.630432 4004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:35.631399 4137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:35.632923 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 4004
I20260811 18:50:35.633038 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:50:35.633255 10076 external_mini_cluster.cc:1428] Setting key 7e4e661bb55267dbc607e0ad67786bf8
I20260811 18:50:35.633342 4137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.215.61@KRBTEST.COM; defaulting to no policy
I20260811 18:50:35.644650 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.634195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.9.215.62:37263)
0811 18:50:35.634440 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.634444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.634530 (+ 86us) server_negotiation.cc:407] Connection header received
0811 18:50:35.635265 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.635268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.635293 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.635397 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.635977 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.637166 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.638122 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.638644 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.638771 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.641222 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.641232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.641232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.641240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.642902 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.643360 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.643363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.643363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.643450 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.643711 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.643711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.643712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.643943 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.644004 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.644273 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.644412 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.61@KRBTEST.COM" created.
I20260811 18:50:35.644658 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.634156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37263 (local address 127.0.0.1:40846)
0811 18:50:35.634382 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.634434 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.635048 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.635407 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.635410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.635713 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.635882 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.635887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.637263 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.637264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.638026 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.638029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.638773 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.638774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.638850 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.639511 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.639522 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.641124 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.643006 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.643016 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.643019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.643283 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.643543 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.643544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.643544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.643656 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.644012 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.644021 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.644109 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.644228 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.644544 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":124,"thread_start_us":123,"threads_started":1}
I20260811 18:50:35.645816 4137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:35.646190 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.646060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37852)
0811 18:50:35.646095 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.646162 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260811 18:50:35.646363 4041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
I20260811 18:50:35.646889 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.646797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37858)
0811 18:50:35.646839 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.646865 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260811 18:50:35.647046 4137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:44099: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab.
Entry for principal kudu/127.9.215.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab.
I20260811 18:50:35.663691 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:44099
--webserver_interface=127.9.215.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab
--principal=kudu/127.9.215.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37263,127.9.215.61:44099,127.9.215.60:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:35.696347 4137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } attempt: 1
I20260811 18:50:35.696939 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.696781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37860)
0811 18:50:35.696854 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.696902 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:50:35.697322 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.697228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37874)
0811 18:50:35.697270 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.697297 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 18:50:35.697418 4137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:44099: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
W20260811 18:50:35.750627 4147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:35.750876 4147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:35.750911 4147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:35.752285 4147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:35.752343 4147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:35.752362 4147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:35.752380 4147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:35.752395 4147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:35.754112 4147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37263,127.9.215.61:44099,127.9.215.60:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab
--principal=kudu/127.9.215.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.215.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:35.754410 4147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:35.754643 4147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:35.759881 4156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:35.759876 4155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:35.759876 4158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:35.760349 4147 server_base.cc:1034] running on GCE node
I20260811 18:50:35.760592 4147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:35.760929 4147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:35.762137 4147 hybrid_clock.cc:648] HybridClock initialized: now 1786474235762113 us; error 41 us; skew 500 ppm
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:35.765465 4147 init.cc:377] Logged in from keytab as kudu/127.9.215.61@KRBTEST.COM (short username kudu)
I20260811 18:50:35.765589 4137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } attempt: 2
I20260811 18:50:35.766065 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.765907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37878)
0811 18:50:35.765975 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.766019 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 18:50:35.766098 4147 webserver.cc:460] Webserver started at http://127.9.215.61:45475/ using document root <none> and password file <none>
I20260811 18:50:35.766335 4147 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:35.766398 4147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:35.766445 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.766344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37886)
0811 18:50:35.766392 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.766424 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 18:50:35.766556 4147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
W20260811 18:50:35.766541 4137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:44099: Network error: Client connection negotiation failed: client connection to 127.9.215.61:44099: connect: Connection refused (error 111)
I20260811 18:50:35.767753 4147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data/instance:
uuid: "73abef1a7f3a49329e294cf6b001afe0"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "10bd50db90d5be4f318b020dbb606452"
server_key_iv: "8b0245239dc585bad7fb48ad7385906c"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.768088 4147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal/instance:
uuid: "73abef1a7f3a49329e294cf6b001afe0"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "10bd50db90d5be4f318b020dbb606452"
server_key_iv: "8b0245239dc585bad7fb48ad7385906c"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.769780 4147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:35.770733 4178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:35.770943 4147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:35.771049 4147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal
uuid: "73abef1a7f3a49329e294cf6b001afe0"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "10bd50db90d5be4f318b020dbb606452"
server_key_iv: "8b0245239dc585bad7fb48ad7385906c"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.771126 4147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:35.789700 4147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:35.792120 4147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:35.792281 4147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:35.796687 4147 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.61:44099
I20260811 18:50:35.796801 4290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.61:44099 every 8 connection(s)
I20260811 18:50:35.797151 4147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
I20260811 18:50:35.798066 4291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:35.799751 4291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:35.800408 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 4147
I20260811 18:50:35.800532 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-1/wal/instance
I20260811 18:50:35.800743 10076 external_mini_cluster.cc:1428] Setting key 3a977af1baff94651ba12827914a4e78
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.215.60@KRBTEST.COM; defaulting to no policy
I20260811 18:50:35.809715 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.800609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.9.215.62:37263)
0811 18:50:35.800671 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.800673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.800946 (+ 273us) server_negotiation.cc:407] Connection header received
0811 18:50:35.801520 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.801522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.801538 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.801637 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.802331 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.802837 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.803907 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.804125 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.804184 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.806625 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.806626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.806626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.806636 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.808236 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.808683 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.808685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.808686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.808752 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.808988 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.808989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.808990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.809120 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.809176 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.809545 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.809636 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:50:35.810039 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.800574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37263 (local address 127.0.0.1:40860)
0811 18:50:35.800793 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.800852 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.801408 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.801727 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.801730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.802005 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.802231 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.802236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.802929 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.802930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.803822 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.803825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.804206 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.804207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.804343 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.804860 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.804870 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.806516 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.808344 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.808353 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.808356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.808596 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.808824 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.808825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.808825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.808936 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.809209 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.809218 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.809378 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.809494 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.809762 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
I20260811 18:50:35.810887 4291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.60@KRBTEST.COM" created.
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:50:35.818418 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.811170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37902)
0811 18:50:35.811213 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.811231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.811285 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.811585 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.811586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.811753 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.811827 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.811830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.812756 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.812757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.813260 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.813262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.813746 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.813748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.813815 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.814238 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.814242 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.815521 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.817342 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.817343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.817344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.817512 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.817734 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.817734 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.817735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.817789 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.818058 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.818059 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.818117 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.818178 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.818294 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 18:50:35.818449 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.811229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.9.215.61:44099)
0811 18:50:35.811405 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.811409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.811427 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:50:35.811445 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.811447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.811498 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.811579 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.811906 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.812673 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.813328 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.813674 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.813729 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.815595 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.815605 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.815606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.815613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.817245 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.817616 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.817619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.817619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.817674 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.817838 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.817839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.817839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.818000 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.818074 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.818218 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.818349 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260811 18:50:35.819099 4291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:35.819514 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.819368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50042)
0811 18:50:35.819412 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.819480 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 18:50:35.819639 4183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
I20260811 18:50:35.820050 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.819963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50054)
0811 18:50:35.820001 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.820026 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260811 18:50:35.820169 4291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:33707: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab.
Entry for principal kudu/127.9.215.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab.
I20260811 18:50:35.830523 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:33707
--webserver_interface=127.9.215.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab
--principal=kudu/127.9.215.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37263,127.9.215.61:44099,127.9.215.60:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:35.857713 4137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } attempt: 3
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:50:35.866518 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.858162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37910)
0811 18:50:35.858252 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.858267 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.858417 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.858714 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.858715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.858892 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.858972 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.858974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.859977 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.859979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.860637 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.860642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.861126 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.861127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.861197 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.861759 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.861765 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.863325 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.865380 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.865381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.865381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.865600 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.865827 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.865828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.865828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.865896 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.866189 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.866190 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.866262 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.866331 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.866435 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 18:50:35.866561 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.858264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37910 (local address 127.9.215.61:44099)
0811 18:50:35.858447 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.858449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.858465 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:50:35.858572 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.858573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.858589 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.858688 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.859049 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.859868 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.860730 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.861049 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.861138 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.863423 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.863426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.863426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.863435 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.865190 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.865679 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.865681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.865681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.865757 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.865949 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.865950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.865950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.866097 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.866162 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.866369 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.866464 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260811 18:50:35.867094 4137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:35.867486 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.867360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50070)
0811 18:50:35.867396 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.867431 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 18:50:35.867842 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.867757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50074)
0811 18:50:35.867796 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.867822 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 18:50:35.867929 4137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:33707: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
I20260811 18:50:35.869421 4291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } attempt: 1
I20260811 18:50:35.869891 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.869757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50086)
0811 18:50:35.869826 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.869860 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:50:35.870215 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.870123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50092)
0811 18:50:35.870159 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.870184 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260811 18:50:35.870301 4291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:33707: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
I20260811 18:50:35.899111 4137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } attempt: 1
I20260811 18:50:35.899673 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.899549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50106)
0811 18:50:35.899605 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.899639 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 18:50:35.900068 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.899924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50114)
0811 18:50:35.899985 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.900034 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 18:50:35.900180 4137 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:33707: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
W20260811 18:50:35.921384 4300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:35.921634 4300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:35.921658 4300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:35.922883 4300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:35.922936 4300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:35.922950 4300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:35.922960 4300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:35.922971 4300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:35.924665 4300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37263,127.9.215.61:44099,127.9.215.60:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab
--principal=kudu/127.9.215.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.215.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:35.924959 4300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:35.925201 4300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:35.930459 4307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:35.930572 4300 server_base.cc:1034] running on GCE node
W20260811 18:50:35.930461 4311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:35.930459 4314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:35.930992 4300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:35.931336 4300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:35.932513 4300 hybrid_clock.cc:648] HybridClock initialized: now 1786474235932512 us; error 51 us; skew 500 ppm
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:35.935285 4300 init.cc:377] Logged in from keytab as kudu/127.9.215.60@KRBTEST.COM (short username kudu)
I20260811 18:50:35.935828 4300 webserver.cc:460] Webserver started at http://127.9.215.60:43917/ using document root <none> and password file <none>
I20260811 18:50:35.936041 4300 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:35.936088 4300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:35.936196 4300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:35.937350 4300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data/instance:
uuid: "2f1558cc155c4329b5ff90995c31682f"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "322aa6fce17f2df705559f28caa537ac"
server_key_iv: "4ad49d218e750282ab5a82e3725613d9"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.937695 4300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal/instance:
uuid: "2f1558cc155c4329b5ff90995c31682f"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "322aa6fce17f2df705559f28caa537ac"
server_key_iv: "4ad49d218e750282ab5a82e3725613d9"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.938452 4291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } attempt: 2
I20260811 18:50:35.938874 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.938741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50128)
0811 18:50:35.938798 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.938837 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 18:50:35.939168 4300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:50:35.939260 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.939173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50144)
0811 18:50:35.939209 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.939235 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 18:50:35.939342 4291 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:33707: Network error: Client connection negotiation failed: client connection to 127.9.215.60:33707: connect: Connection refused (error 111)
I20260811 18:50:35.940047 4321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:35.940194 4300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:35.940289 4300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal
uuid: "2f1558cc155c4329b5ff90995c31682f"
format_stamp: "Formatted at 2026-08-11 18:50:35 on dist-test-slave-pww0"
server_key: "322aa6fce17f2df705559f28caa537ac"
server_key_iv: "4ad49d218e750282ab5a82e3725613d9"
server_key_version: "encryptionkey@0"
I20260811 18:50:35.940363 4300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:35.953653 4300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:35.956202 4300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:35.956367 4300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:35.960493 4300 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.60:33707
I20260811 18:50:35.960548 4431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.60:33707 every 8 connection(s)
I20260811 18:50:35.960911 4300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
I20260811 18:50:35.961889 4433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:35.963615 4433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:35.966903 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 4300
I20260811 18:50:35.967021 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474068101683-10076-0/minicluster-data/master-2/wal/instance
I20260811 18:50:35.967175 10076 external_mini_cluster.cc:1428] Setting key 18008cd6cb5507dd2f7fb502e08f1d86
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:35.975319 4137 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } attempt: 2
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:50:35.981158 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.967803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.9.215.60:33707)
0811 18:50:35.968015 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.968021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.968045 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:50:35.968070 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.968072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.968104 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.968330 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.968959 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.972981 (+ 4022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.974151 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.974766 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.974853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.977668 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.977682 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.977682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.977691 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.979614 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.980034 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.980039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.980039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.980102 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.980308 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.980309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.980309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.980478 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.980654 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.980708 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.980854 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":123,"threads_started":1}
I20260811 18:50:35.981201 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.964447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.9.215.62:37263)
0811 18:50:35.964498 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.964500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.964705 (+ 205us) server_negotiation.cc:407] Connection header received
0811 18:50:35.965195 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.965196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.965206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.965294 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.966190 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.966683 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.968848 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.969137 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.969213 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.976051 (+ 6838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.976054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.976054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.976063 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.977850 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.979118 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.979120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.979121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.979193 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.979821 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.979823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.979823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.980001 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.980053 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.980962 (+ 909us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.981076 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:50:35.981585 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.964387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37263 (local address 127.0.0.1:40872)
0811 18:50:35.964579 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.964625 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.965102 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.965474 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.965478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.965804 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.966022 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.966026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.967428 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.967430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.968752 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.968755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.969311 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.969312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.969702 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.973844 (+ 4142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.973859 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.975912 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.978230 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.978240 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.978243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.979001 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.979276 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.979281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.979281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.979498 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.980200 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.980210 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.980549 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.981321 (+ 772us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.981469 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260811 18:50:35.982080 4433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:35.992700 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.979660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50158)
0811 18:50:35.979778 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.979789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.979829 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.981804 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.981805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.981964 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.982021 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.982024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.984113 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.984114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.984867 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.984870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.985334 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.985336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.985392 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.985955 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.985962 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.988889 (+ 2927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.991256 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.991258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.991258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.991586 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.991953 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.991954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.991954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.992041 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.992360 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.992361 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.992416 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.992461 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.992586 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 18:50:35.992740 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.982480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:44099 (local address 127.0.0.1:37914)
0811 18:50:35.982730 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.982739 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.982781 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:35.982943 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:35.982945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:35.983099 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:35.983167 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.983169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.983932 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.983933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.984703 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:35.984715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.985208 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.985209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:35.985273 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.985805 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:35.985815 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:35.987947 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:35.990973 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.990975 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.990975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.991412 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.991683 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:35.991684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:35.991684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:35.991974 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:35.992351 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:35.992353 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:35.992404 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:50:35.992488 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:50:35.992599 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
I20260811 18:50:35.993080 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.979905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.9.215.60:33707)
0811 18:50:35.981663 (+ 1758us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.981665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.981673 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:50:35.981694 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.981694 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.981708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.982089 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.982118 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.984024 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.984953 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.985246 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.985515 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.988991 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.988993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.988994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.989003 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.991126 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.991678 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.991680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.991680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.991767 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.992129 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.992131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.992131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.992278 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.992850 (+ 572us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.992897 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.992986 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1741,"thread_start_us":141,"threads_started":1}
I20260811 18:50:35.993207 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.982710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37914 (local address 127.9.215.61:44099)
0811 18:50:35.982757 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.982759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.982767 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:50:35.982836 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.982837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.982845 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.982892 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.983278 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.983851 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.984812 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.985092 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.985150 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.988093 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.988096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.988096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.988131 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.990496 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.991532 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.991533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.991534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.991599 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.992076 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.992078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.992079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.992260 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.992323 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.992961 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.993060 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:50:35.994442 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.982580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.9.215.62:37263)
0811 18:50:35.982616 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.982618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.982689 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:50:35.982795 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.982796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.982805 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.982831 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.983517 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.984295 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.985384 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.986222 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.986284 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:35.989628 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:35.989630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:35.989630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:35.989639 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:35.992082 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.992503 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.992504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.992504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.992561 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:35.993650 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:35.993652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:35.993652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:35.993813 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:35.993990 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:35.994249 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 18:50:35.994344 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 18:50:35.995707 4433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:50:35.995806 4137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f: Bootstrap starting.
I20260811 18:50:35.996551 4137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:35.996865 4137 log.cc:824] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:35.997965 4137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f: No bootstrap required, opened a new log
I20260811 18:50:35.999747 4137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:35.999878 4137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:35.999912 4137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e2447e7e0f44888db849fdc3118d4f, State: Initialized, Role: FOLLOWER
I20260811 18:50:36.000078 4137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.000453 4446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.000563 4446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:36.000469 4137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 18:50:35 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:50:36.005275 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.995265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.9.215.61:44099)
0811 18:50:35.995307 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:35.995309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:35.995317 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:50:35.995341 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:35.995342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:35.995352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:35.995412 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:35.996138 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.996906 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.997973 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:35.998283 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:35.998328 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.001403 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:36.001405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:36.001405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:36.001415 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:36.003173 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.003665 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.003667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.003667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.003731 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.003913 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.003914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.003914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.004046 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:36.004823 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:36.005076 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 18:50:36.005178 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 11 18:50:36 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:50:36.003569 4004 master_runner.cc:186] Error getting master registration for 127.9.215.61:44099: OK, instance_id { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" instance_seqno: 1786474235795194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:50:36.007172 4004 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:50:36.007313 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 18:50:36.007762 4464 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:36.007820 4464 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:36.010807 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.997359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50170)
0811 18:50:35.997470 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:35.997481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:35.997523 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:36.002427 (+ 4904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:36.002428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:36.002608 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:36.002674 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.002677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.003555 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.003556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.004267 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.004271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.004701 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.004702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.004749 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.005717 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:36.005722 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:36.007496 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:36.009679 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:36.009680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:36.009680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:36.009889 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:36.010071 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:36.010072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:36.010072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:36.010134 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:36.010510 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:36.010511 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:36.010554 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 18:50:36.010611 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 18:50:36.010709 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 18:50:36.011078 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:35.997437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.9.215.60:33707)
0811 18:50:36.000515 (+ 3078us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.000518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.000530 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:50:36.000557 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.000558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.000571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.002747 (+ 2176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.002772 (+ 25us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.003416 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.004360 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.004631 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.004909 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.007585 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:36.007587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:36.007587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:36.007593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:36.009584 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.009958 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.009960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.009960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.010012 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.010191 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.010191 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.010192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.010410 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:36.010480 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:36.010927 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 18:50:36.011001 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3048}
I20260811 18:50:36.013326 4433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: Bootstrap starting.
I20260811 18:50:36.014345 4433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:36.014707 4433 log.cc:824] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:36.018298 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.008933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.9.215.60:33707)
0811 18:50:36.008987 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.008988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.009899 (+ 911us) server_negotiation.cc:407] Connection header received
0811 18:50:36.009993 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.009994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.010005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.010050 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.010856 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.011622 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.012856 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.013135 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.013182 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.015130 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:36.015132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:36.015132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:36.015140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:36.016952 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.017340 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.017342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.017342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.017402 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.017575 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.017576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.017576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.017692 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:36.017768 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:36.018068 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 18:50:36.018170 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:50:36.024345 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.008958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.9.215.61:44099)
0811 18:50:36.008999 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.009001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.015351 (+ 6350us) server_negotiation.cc:407] Connection header received
0811 18:50:36.015542 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.015543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.015556 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.015590 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.017932 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.018587 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.019257 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.019590 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.019654 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.021306 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:36.021308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:36.021309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:36.021319 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:36.022949 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.023379 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.023381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.023381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.023444 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.023698 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.023699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.023700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.023849 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:36.023925 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:36.024156 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 18:50:36.024248 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:50:36.025251 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.009087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40892 (local address 127.9.215.62:37263)
0811 18:50:36.009139 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.009142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.015240 (+ 6098us) server_negotiation.cc:407] Connection header received
0811 18:50:36.015683 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.015686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.015700 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.015733 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.016953 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.017661 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.019983 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.020269 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.020313 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.022018 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:36.022020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:36.022020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:36.022029 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:36.023916 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.024272 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.024273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.024273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.024331 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.024588 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.024589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.024590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.024722 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:36.024793 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:36.025098 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 18:50:36.025187 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:50:36.025521 4433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: No bootstrap required, opened a new log
I20260811 18:50:36.028144 4433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.028329 4433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:36.028365 4433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1558cc155c4329b5ff90995c31682f, State: Initialized, Role: FOLLOWER
I20260811 18:50:36.028512 4433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.028856 4480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.028918 4433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:36.028952 4480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:36.030470 4291 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } attempt: 3
Aug 11 18:50:36 dist-test-slave-pww0 krb5kdc[3982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474235, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:50:36.034754 4300 master_runner.cc:186] Error getting master registration for 127.9.215.61:44099: OK, instance_id { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" instance_seqno: 1786474235795194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:50:36.037535 4300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:50:36.037837 4511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:36.037892 4511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:36.039254 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.031015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:33707 (local address 127.0.0.1:50194)
0811 18:50:36.031211 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:36.031229 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:36.031295 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:36.031624 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:36.031625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:36.031830 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:36.031888 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.031891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.032817 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.032819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.033545 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:36.033548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.034016 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.034017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:36.034072 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.034529 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:36.034535 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:36.036088 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:36.038143 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:36.038145 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:36.038146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:36.038368 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:36.038572 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:36.038573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:36.038573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:36.038631 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:36.038917 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:36.038918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:36.038969 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:50:36.039028 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:50:36.039155 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 18:50:36.039619 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:36.031149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50194 (local address 127.9.215.60:33707)
0811 18:50:36.031276 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:36.031278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:36.031291 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:50:36.031357 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:36.031358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:36.031385 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:36.031451 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:36.031978 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.032716 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.033629 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:36.033899 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:36.033957 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:36.036201 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:36.036203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:36.036203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:36.036211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:36.038028 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.038447 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.038448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.038449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.038508 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:36.038693 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:36.038694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:36.038695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:36.038834 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:36.038887 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:36.039429 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 18:50:36.039541 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 18:50:36.041499 4291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: Bootstrap starting.
I20260811 18:50:36.042332 4291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:36.042675 4291 log.cc:824] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:36.044346 4291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: No bootstrap required, opened a new log
I20260811 18:50:36.046571 4291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.046748 4291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:36.046787 4291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73abef1a7f3a49329e294cf6b001afe0, State: Initialized, Role: FOLLOWER
I20260811 18:50:36.046921 4291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.047322 4291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:36.047273 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.047529 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:50:36.049530 4542 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:36.049652 4542 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:36.052438 4147 master_runner.cc:263] No leader master found from master 73abef1a7f3a49329e294cf6b001afe0
I20260811 18:50:36.052507 4147 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:50:36.103344 4446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:36.103515 4446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.103802 4446 leader_election.cc:288] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73abef1a7f3a49329e294cf6b001afe0 (127.9.215.61:44099), 2f1558cc155c4329b5ff90995c31682f (127.9.215.60:33707)
I20260811 18:50:36.104301 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90e2447e7e0f44888db849fdc3118d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73abef1a7f3a49329e294cf6b001afe0" is_pre_election: true
I20260811 18:50:36.104454 4228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e2447e7e0f44888db849fdc3118d4f in term 0.
I20260811 18:50:36.104643 4384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90e2447e7e0f44888db849fdc3118d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f1558cc155c4329b5ff90995c31682f" is_pre_election: true
I20260811 18:50:36.104799 4384 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90e2447e7e0f44888db849fdc3118d4f in term 0.
I20260811 18:50:36.105007 4037 leader_election.cc:302] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f1558cc155c4329b5ff90995c31682f, 90e2447e7e0f44888db849fdc3118d4f; no voters:
I20260811 18:50:36.105283 4446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:36.105341 4446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:36.105367 4446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:36.106284 4446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.106429 4446 leader_election.cc:288] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [CANDIDATE]: Term 1 election: Requested vote from peers 73abef1a7f3a49329e294cf6b001afe0 (127.9.215.61:44099), 2f1558cc155c4329b5ff90995c31682f (127.9.215.60:33707)
I20260811 18:50:36.106750 4384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90e2447e7e0f44888db849fdc3118d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f1558cc155c4329b5ff90995c31682f"
I20260811 18:50:36.106834 4384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:36.107770 4384 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e2447e7e0f44888db849fdc3118d4f in term 1.
I20260811 18:50:36.107945 4037 leader_election.cc:302] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f1558cc155c4329b5ff90995c31682f, 90e2447e7e0f44888db849fdc3118d4f; no voters:
I20260811 18:50:36.108057 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90e2447e7e0f44888db849fdc3118d4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73abef1a7f3a49329e294cf6b001afe0"
I20260811 18:50:36.108142 4228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:36.109067 4228 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90e2447e7e0f44888db849fdc3118d4f in term 1.
I20260811 18:50:36.109287 4446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:36.109479 4446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [term 1 LEADER]: Becoming Leader. State: Replica: 90e2447e7e0f44888db849fdc3118d4f, State: Running, Role: LEADER
I20260811 18:50:36.109597 4446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } }
I20260811 18:50:36.110018 4446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e2447e7e0f44888db849fdc3118d4f. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.110095 4446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:36.113207 4551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:36.113462 4551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:36.115170 4384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [term 1 FOLLOWER]: Refusing update from remote peer 90e2447e7e0f44888db849fdc3118d4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:36.115576 4446 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:36.115715 4228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [term 1 FOLLOWER]: Refusing update from remote peer 90e2447e7e0f44888db849fdc3118d4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:36.117169 4480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e2447e7e0f44888db849fdc3118d4f. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.117278 4480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:36.117337 4446 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:36.118721 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e2447e7e0f44888db849fdc3118d4f. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.118824 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:36.125937 4446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.126096 4446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:36.125937 4561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.126190 4561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:36.126370 4480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.126466 4480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:36.127030 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90e2447e7e0f44888db849fdc3118d4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e2447e7e0f44888db849fdc3118d4f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37263 } } peers { permanent_uuid: "73abef1a7f3a49329e294cf6b001afe0" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 44099 } } peers { permanent_uuid: "2f1558cc155c4329b5ff90995c31682f" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 33707 } } }
I20260811 18:50:36.127140 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:50:36.127316 4555 mvcc.cc:205] Tried to move back new op lower bound from 7317398471125311488 to 7317398471105523712. Current Snapshot: MvccSnapshot[applied={T|T < 7317398471125311488 or (T in {7317398471125311488})}]
I20260811 18:50:36.127427 4551 catalog_manager.cc:1347] Generated new cluster ID: df76969e9fca48129445a552d9952b44
I20260811 18:50:36.127480 4551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:36.127758 4556 mvcc.cc:205] Tried to move back new op lower bound from 7317398471125311488 to 7317398471105523712. Current Snapshot: MvccSnapshot[applied={T|T < 7317398471125311488}]
I20260811 18:50:36.145857 4551 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:36.146679 4551 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:36.154488 4551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90e2447e7e0f44888db849fdc3118d4f: Generated new TSK 0
I20260811 18:50:36.154760 4551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:37.038401 4511 catalog_manager.cc:1259] Loaded cluster ID: df76969e9fca48129445a552d9952b44
I20260811 18:50:37.038491 4511 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: loading cluster ID for follower catalog manager: success
I20260811 18:50:37.039361 4511 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: acquiring CA information for follower catalog manager: success
I20260811 18:50:37.039737 4511 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f1558cc155c4329b5ff90995c31682f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:50:37.050185 4542 catalog_manager.cc:1259] Loaded cluster ID: df76969e9fca48129445a552d9952b44
I20260811 18:50:37.050297 4542 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: loading cluster ID for follower catalog manager: success
I20260811 18:50:37.051214 4542 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: acquiring CA information for follower catalog manager: success
I20260811 18:50:37.051591 4542 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73abef1a7f3a49329e294cf6b001afe0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:50:37.217708 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 4004
I20260811 18:50:37.235880 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 4147
I20260811 18:50:37.241873 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 4300
2026-08-11T18:50:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3099 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:37 dist-test-slave-pww0 krb5kdc[4582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:37 dist-test-slave-pww0 krb5kdc[4582](info): set up 2 sockets
Aug 11 18:50:37 dist-test-slave-pww0 krb5kdc[4582](info): commencing operation
krb5kdc: starting...
W20260811 18:50:38.466655 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:50:38 dist-test-slave-pww0 krb5kdc[4582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:50:38.598173 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36185
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:38.683786 4604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:38.684006 4604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:38.684031 4604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:38.685237 4604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:38.685285 4604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:38.685299 4604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:38.685310 4604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:38.685321 4604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:38.686923 4604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:38.687186 4604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:38.687427 4604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:38.693153 4609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:38.693166 4622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:38.693188 4604 server_base.cc:1034] running on GCE node
W20260811 18:50:38.693171 4612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:38.693766 4604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:38.694092 4604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:38.695261 4604 hybrid_clock.cc:648] HybridClock initialized: now 1786474238695236 us; error 34 us; skew 500 ppm
Aug 11 18:50:38 dist-test-slave-pww0 krb5kdc[4582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:38.698141 4604 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:38.698619 4604 webserver.cc:460] Webserver started at http://127.9.215.62:44303/ using document root <none> and password file <none>
I20260811 18:50:38.698858 4604 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:38.698909 4604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:38.699034 4604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:38.700167 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "fd79e4ac519745b9aca0dc673884b764"
format_stamp: "Formatted at 2026-08-11 18:50:38 on dist-test-slave-pww0"
server_key: "8c5d275ffa9a87c8fbf7186275062c3f"
server_key_iv: "cd90163de84d02b00d3146a295d2f437"
server_key_version: "encryptionkey@0"
I20260811 18:50:38.700506 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "fd79e4ac519745b9aca0dc673884b764"
format_stamp: "Formatted at 2026-08-11 18:50:38 on dist-test-slave-pww0"
server_key: "8c5d275ffa9a87c8fbf7186275062c3f"
server_key_iv: "cd90163de84d02b00d3146a295d2f437"
server_key_version: "encryptionkey@0"
I20260811 18:50:38.701874 4604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:38.702696 4637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:38.702862 4604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:38.702942 4604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "fd79e4ac519745b9aca0dc673884b764"
format_stamp: "Formatted at 2026-08-11 18:50:38 on dist-test-slave-pww0"
server_key: "8c5d275ffa9a87c8fbf7186275062c3f"
server_key_iv: "cd90163de84d02b00d3146a295d2f437"
server_key_version: "encryptionkey@0"
I20260811 18:50:38.703009 4604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:38.719403 4604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:38.721714 4604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:38.721879 4604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:38.726081 4604 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36185
I20260811 18:50:38.726153 4744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36185 every 8 connection(s)
I20260811 18:50:38.726500 4604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:38.727556 4746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:38.729931 4746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764: Bootstrap starting.
I20260811 18:50:38.730564 4746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:38.730852 4746 log.cc:824] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:38.731685 4746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764: No bootstrap required, opened a new log
I20260811 18:50:38.733386 4746 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd79e4ac519745b9aca0dc673884b764" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36185 } }
I20260811 18:50:38.733537 4746 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:38.733560 4746 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd79e4ac519745b9aca0dc673884b764, State: Initialized, Role: FOLLOWER
I20260811 18:50:38.733728 4746 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd79e4ac519745b9aca0dc673884b764" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36185 } }
I20260811 18:50:38.733796 4746 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:38.733819 4746 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:38.733870 4746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:38.734305 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 4604
I20260811 18:50:38.734392 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:50:38.734469 4746 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd79e4ac519745b9aca0dc673884b764" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36185 } }
I20260811 18:50:38.734557 10076 external_mini_cluster.cc:1428] Setting key a6770d75d0b0ade2d1dd32485f2c0615
I20260811 18:50:38.734581 4746 leader_election.cc:302] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd79e4ac519745b9aca0dc673884b764; no voters:
I20260811 18:50:38.734766 4746 leader_election.cc:288] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:38.734822 4752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:38.734994 4752 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [term 1 LEADER]: Becoming Leader. State: Replica: fd79e4ac519745b9aca0dc673884b764, State: Running, Role: LEADER
I20260811 18:50:38.735075 4746 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:38.735153 4752 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd79e4ac519745b9aca0dc673884b764" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36185 } }
I20260811 18:50:38.735527 4752 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd79e4ac519745b9aca0dc673884b764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd79e4ac519745b9aca0dc673884b764" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36185 } } }
I20260811 18:50:38.735661 4752 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:38.735836 4752 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd79e4ac519745b9aca0dc673884b764. Latest consensus state: current_term: 1 leader_uuid: "fd79e4ac519745b9aca0dc673884b764" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd79e4ac519745b9aca0dc673884b764" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36185 } } }
I20260811 18:50:38.735942 4752 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:38.736284 4771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:38.736832 4771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:38.739550 4771 catalog_manager.cc:1347] Generated new cluster ID: ece6a62ed8504c429dc77ca7d6e5db02
I20260811 18:50:38.739612 4771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:50:38 dist-test-slave-pww0 krb5kdc[4582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:38.747062 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:38.735262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.9.215.62:36185)
0811 18:50:38.735428 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:38.735438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:38.735494 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:50:38.735653 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:38.735661 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:38.735728 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:38.736719 (+ 991us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:38.736847 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.738227 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.739032 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:38.739745 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:38.739867 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:38.742659 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:38.742664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:38.742665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:38.742679 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:38.745290 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:38.745766 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:38.745769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:38.745770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:38.745884 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:38.746070 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:38.746071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:38.746071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:38.746398 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:38.746552 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:38.746665 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 18:50:38.746809 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260811 18:50:38.748139 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:50:38.761847 4771 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:38.762434 4771 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:38.766808 4771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd79e4ac519745b9aca0dc673884b764: Generated new TSK 0
I20260811 18:50:38.767005 4771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:50:38.802170 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 4604
2026-08-11T18:50:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1572 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:38 dist-test-slave-pww0 krb5kdc[4796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:38 dist-test-slave-pww0 krb5kdc[4796](info): set up 2 sockets
Aug 11 18:50:38 dist-test-slave-pww0 krb5kdc[4796](info): commencing operation
krb5kdc: starting...
W20260811 18:50:40.052078 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:50:40.182647 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43843
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:43843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:40.268260 4829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:40.268481 4829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:40.268504 4829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:40.269729 4829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:40.269773 4829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:40.269786 4829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:40.269797 4829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:40.269809 4829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:40.271355 4829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:43843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:40.271636 4829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:40.271857 4829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:40.276966 4843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.276971 4839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.276971 4838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.277010 4829 server_base.cc:1034] running on GCE node
I20260811 18:50:40.277452 4829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.277756 4829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.278911 4829 hybrid_clock.cc:648] HybridClock initialized: now 1786474240278900 us; error 29 us; skew 500 ppm
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:40.285665 4829 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:40.286227 4829 webserver.cc:460] Webserver started at http://127.9.215.62:33795/ using document root <none> and password file <none>
I20260811 18:50:40.286448 4829 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.286502 4829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.286644 4829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.287842 4829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "c119efe934de493c816f3787ee917074"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "f01614ed02b798e2d51cc021e0714792"
server_key_iv: "9493d1929a5877c4d5590a8cda9a5241"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.288218 4829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "c119efe934de493c816f3787ee917074"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "f01614ed02b798e2d51cc021e0714792"
server_key_iv: "9493d1929a5877c4d5590a8cda9a5241"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.289561 4829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:40.290284 4859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.290473 4829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:40.290570 4829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c119efe934de493c816f3787ee917074"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "f01614ed02b798e2d51cc021e0714792"
server_key_iv: "9493d1929a5877c4d5590a8cda9a5241"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.290637 4829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.307024 4829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.309458 4829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.309652 4829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.314278 4829 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:43843
I20260811 18:50:40.314338 4977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:43843 every 8 connection(s)
I20260811 18:50:40.314698 4829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:40.315771 4978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:40.318228 4978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap starting.
I20260811 18:50:40.318826 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 4829
I20260811 18:50:40.318845 4978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:40.318928 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:50:40.319139 10076 external_mini_cluster.cc:1428] Setting key da3c3ec7289db2c8ff36ea0bca5b6db8
I20260811 18:50:40.319139 4978 log.cc:824] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:40.320037 4978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: No bootstrap required, opened a new log
I20260811 18:50:40.322559 4978 raft_consensus.cc:348] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:40.322757 4978 raft_consensus.cc:374] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:40.322795 4978 raft_consensus.cc:729] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c119efe934de493c816f3787ee917074, State: Initialized, Role: FOLLOWER
I20260811 18:50:40.322969 4978 consensus_queue.cc:260] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:40.323055 4978 raft_consensus.cc:388] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:40.323087 4978 raft_consensus.cc:482] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:40.323135 4978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:40.323803 4978 raft_consensus.cc:504] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:40.323920 4978 leader_election.cc:302] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c119efe934de493c816f3787ee917074; no voters:
I20260811 18:50:40.324095 4978 leader_election.cc:288] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:40.324172 4986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:40.324363 4978 sys_catalog.cc:564] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:40.324404 4986 raft_consensus.cc:686] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 LEADER]: Becoming Leader. State: Replica: c119efe934de493c816f3787ee917074, State: Running, Role: LEADER
I20260811 18:50:40.324906 4986 consensus_queue.cc:237] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:40.326591 4995 sys_catalog.cc:455] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c119efe934de493c816f3787ee917074. Latest consensus state: current_term: 1 leader_uuid: "c119efe934de493c816f3787ee917074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } } }
I20260811 18:50:40.326586 4993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c119efe934de493c816f3787ee917074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } } }
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:40.326699 4995 sys_catalog.cc:458] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:40.326704 4993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:40.327024 5016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:40.327750 5016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:40.329488 5016 catalog_manager.cc:1347] Generated new cluster ID: 2699fb79251241878f017c698dda819a
I20260811 18:50:40.329561 5016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:40.331045 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.319720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.9.215.62:43843)
0811 18:50:40.319918 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.319929 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.319964 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:50:40.320528 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.320538 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.320609 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.320769 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.321382 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.322756 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.323747 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.324427 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.324543 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.327512 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.327518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.327518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:40.327532 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.329497 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.329863 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.329866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.329867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.329953 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.330141 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.330142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.330142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.330412 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.330546 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.330681 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.330841 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260811 18:50:40.356011 5016 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:40.356567 5016 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:40.358661 5016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Generated new TSK 0
I20260811 18:50:40.358837 5016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:50:40.390740 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43843
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:40.498623 5026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:40.498864 5026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:40.498891 5026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:40.500172 5026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:40.500227 5026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:40.500257 5026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:50:40.501839 5026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:40.502107 5026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:40.502351 5026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:40.507826 5032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.507826 5031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.507824 5036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.507892 5026 server_base.cc:1034] running on GCE node
I20260811 18:50:40.508212 5026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.508474 5026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.509644 5026 hybrid_clock.cc:648] HybridClock initialized: now 1786474240509621 us; error 39 us; skew 500 ppm
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:40.512444 5026 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:50:40.512921 5026 webserver.cc:460] Webserver started at http://127.9.215.1:45515/ using document root <none> and password file <none>
I20260811 18:50:40.513155 5026 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.513206 5026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.513370 5026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.514441 5026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "8972bec699c940d4b10f4099542ea1a7"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "ff39850402e282e300b0effd40a43113"
server_key_iv: "1dc6a5b9801242a025f105b73ae5a19f"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.514777 5026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "8972bec699c940d4b10f4099542ea1a7"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "ff39850402e282e300b0effd40a43113"
server_key_iv: "1dc6a5b9801242a025f105b73ae5a19f"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.516192 5026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:50:40.517014 5051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.517225 5026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:40.517326 5026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "8972bec699c940d4b10f4099542ea1a7"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "ff39850402e282e300b0effd40a43113"
server_key_iv: "1dc6a5b9801242a025f105b73ae5a19f"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.517390 5026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.535589 5026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.537879 5026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.538427 5026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.538727 5026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:40.539055 5026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:40.539098 5026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.539143 5026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:40.539168 5026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.544955 5026 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:38927
I20260811 18:50:40.545048 5259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:38927 every 8 connection(s)
I20260811 18:50:40.545343 5026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:40.548013 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 5026
I20260811 18:50:40.548143 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:40.548352 10076 external_mini_cluster.cc:1428] Setting key d513af2e28c8a8c92a9ac5d76a8e1b39
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:40.555781 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.546141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:38641 (local address 127.9.215.62:43843)
0811 18:50:40.546231 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.546233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.546361 (+ 128us) server_negotiation.cc:407] Connection header received
0811 18:50:40.546920 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.546922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.546935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.546987 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.548174 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.548799 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.549855 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.550132 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.550213 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.552772 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.552774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.552774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:40.552782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.554099 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.554613 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.554614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.554614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.554679 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.554963 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.554963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.554964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.555089 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.555138 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.555585 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.555691 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:50:40.555950 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.546029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:38641)
0811 18:50:40.546208 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.546275 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.546807 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.547061 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.547064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.547735 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:40.548038 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.548047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.548887 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.548887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.549765 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.549767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.550239 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.550240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.550376 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.550958 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:40.550969 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:40.552649 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:40.554185 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:40.554194 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:40.554196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:40.554510 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:40.554750 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:40.554751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:40.554751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:40.554902 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:40.555147 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:40.555154 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:40.555343 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.555501 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.555699 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260811 18:50:40.556579 5260 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43843
I20260811 18:50:40.556695 5260 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:40.556891 5260 heartbeater.cc:511] Master 127.9.215.62:43843 requested a full tablet report, sending...
I20260811 18:50:40.557497 4896 ts_manager.cc:194] Registered new tserver with Master: 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927)
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:50:40.558423 4896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:38641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:50:40.578907 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43843
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:40.664314 5268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:40.664546 5268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:40.664575 5268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:40.665891 5268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:40.665944 5268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:40.665974 5268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:50:40.667680 5268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:40.667946 5268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:40.668182 5268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:50:40.673543 5268 server_base.cc:1034] running on GCE node
W20260811 18:50:40.673497 5283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.673498 5277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.673641 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.674021 5268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.674309 5268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.675518 5268 hybrid_clock.cc:648] HybridClock initialized: now 1786474240675490 us; error 36 us; skew 500 ppm
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:40.678421 5268 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:50:40.678886 5268 webserver.cc:460] Webserver started at http://127.9.215.2:37823/ using document root <none> and password file <none>
I20260811 18:50:40.679098 5268 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.679144 5268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.679260 5268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.680380 5268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "7db4627535f84ab28d28d6e3f76630e8"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "17c0f9efb3cd035901d0f2be352858b4"
server_key_iv: "45013b98b6507c54dd384c2dae91e2de"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.680714 5268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "7db4627535f84ab28d28d6e3f76630e8"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "17c0f9efb3cd035901d0f2be352858b4"
server_key_iv: "45013b98b6507c54dd384c2dae91e2de"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.682111 5268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:40.682936 5299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.683108 5268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:40.683198 5268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "7db4627535f84ab28d28d6e3f76630e8"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "17c0f9efb3cd035901d0f2be352858b4"
server_key_iv: "45013b98b6507c54dd384c2dae91e2de"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.683279 5268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.697592 5268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.700139 5268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.700281 5268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.700570 5268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:40.701007 5268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:40.701073 5268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.701109 5268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:40.701128 5268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.707392 5268 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:38119
I20260811 18:50:40.707446 5517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:38119 every 8 connection(s)
I20260811 18:50:40.707835 5268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:40.715219 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 5268
I20260811 18:50:40.715339 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:40.715574 10076 external_mini_cluster.cc:1428] Setting key 3dead3c599e729732bfad8941f02729e
I20260811 18:50:40.718900 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.708624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:52059 (local address 127.9.215.62:43843)
0811 18:50:40.708690 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.708693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.708924 (+ 231us) server_negotiation.cc:407] Connection header received
0811 18:50:40.709469 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.709471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.709485 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.709540 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.710575 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.711073 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.712176 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.712372 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.712431 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.714832 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.714834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.714834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:40.714842 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.716912 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.717508 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.717509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.717509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.717579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.717962 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.717963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.717963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.718100 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.718180 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.718708 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.718800 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:50:40.719352 5523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.708629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:52059)
0811 18:50:40.708798 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.708838 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.709362 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.709618 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.709622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.710235 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:40.710467 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.710474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.711169 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.711171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.712073 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.712077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.712453 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.712454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.712582 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.713141 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:40.713150 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:40.714688 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:40.717022 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:40.717037 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:40.717042 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:40.717405 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:40.717655 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:40.717656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:40.717657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:40.717822 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:40.718183 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:40.718191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:40.718384 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.718624 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.718846 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260811 18:50:40.719954 5520 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43843
I20260811 18:50:40.720058 5520 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:40.720299 5520 heartbeater.cc:511] Master 127.9.215.62:43843 requested a full tablet report, sending...
I20260811 18:50:40.720706 4896 ts_manager.cc:194] Registered new tserver with Master: 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
I20260811 18:50:40.721076 4896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:52059
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:50:40.747068 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43843
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:40.830981 5528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:40.831208 5528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:40.831244 5528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:40.832448 5528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:40.832494 5528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:40.832532 5528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:50:40.834065 5528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:40.834313 5528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:40.834542 5528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:40.840255 5537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.840287 5528 server_base.cc:1034] running on GCE node
W20260811 18:50:40.840246 5534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:40.840246 5535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:40.840651 5528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:40.840914 5528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:40.842085 5528 hybrid_clock.cc:648] HybridClock initialized: now 1786474240842068 us; error 36 us; skew 500 ppm
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:40.844764 5528 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:50:40.845288 5528 webserver.cc:460] Webserver started at http://127.9.215.3:39103/ using document root <none> and password file <none>
I20260811 18:50:40.845501 5528 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:40.845549 5528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:40.845690 5528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:40.846773 5528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "3b7445029c454defa4e58fe47fb172e0"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "e82fca9dacf7cf5625848f11b02c187a"
server_key_iv: "1f953f46d7a0c84c1a97f2d5ea643e73"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.847124 5528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "3b7445029c454defa4e58fe47fb172e0"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "e82fca9dacf7cf5625848f11b02c187a"
server_key_iv: "1f953f46d7a0c84c1a97f2d5ea643e73"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.848625 5528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:50:40.849435 5555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.849718 5528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:40.849815 5528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "3b7445029c454defa4e58fe47fb172e0"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "e82fca9dacf7cf5625848f11b02c187a"
server_key_iv: "1f953f46d7a0c84c1a97f2d5ea643e73"
server_key_version: "encryptionkey@0"
I20260811 18:50:40.849886 5528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:40.859990 5528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:40.862293 5528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:40.862447 5528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:40.862697 5528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:40.863009 5528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:40.863045 5528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.863075 5528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:40.863093 5528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:40.869112 5528 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44057
I20260811 18:50:40.869160 5763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44057 every 8 connection(s)
I20260811 18:50:40.869505 5528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:40.873279 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 5528
I20260811 18:50:40.873385 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:40.873575 10076 external_mini_cluster.cc:1428] Setting key c205e0b786dde57c0faea53b9a063250
Aug 11 18:50:40 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:40.879164 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.870294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:54417 (local address 127.9.215.62:43843)
0811 18:50:40.870364 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.870367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.870568 (+ 201us) server_negotiation.cc:407] Connection header received
0811 18:50:40.871043 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.871044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.871057 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.871110 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.872121 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.872636 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.873902 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.874136 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.874226 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.876402 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.876405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.876405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:40.876413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.877794 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.878235 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.878236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.878237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.878298 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.878524 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.878525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.878525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.878656 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.878713 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.878983 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.879068 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:50:40.879365 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.870236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:54417)
0811 18:50:40.870415 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.870460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.870961 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.871174 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.871178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.871805 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:40.872011 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.872017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.872725 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.872725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.873816 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.873819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.874240 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.874241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.874391 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.874904 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:40.874913 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:40.876300 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:40.877890 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:40.877899 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:40.877902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:40.878158 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:40.878354 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:40.878354 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:40.878355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:40.878467 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:40.878710 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:40.878718 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:40.878864 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.878956 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.879133 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260811 18:50:40.879928 5764 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43843
I20260811 18:50:40.880024 5764 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:40.880211 5764 heartbeater.cc:511] Master 127.9.215.62:43843 requested a full tablet report, sending...
I20260811 18:50:40.880636 4896 ts_manager.cc:194] Registered new tserver with Master: 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057)
I20260811 18:50:40.880995 4896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:54417
I20260811 18:50:40.885340 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:40.892639 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.886671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.9.215.62:43843)
0811 18:50:40.886728 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.886730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.886806 (+ 76us) server_negotiation.cc:407] Connection header received
0811 18:50:40.886867 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.886868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.886880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.886940 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:40.887558 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.888050 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.888809 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.888994 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.889044 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.890225 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:40.890227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:40.890227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:40.890235 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:40.891523 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.891884 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.891885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.891885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.891952 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:40.892131 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:40.892132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:40.892132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:40.892278 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:40.892353 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.892491 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.892549 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:50:40.894506 4896 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:40.896276 4896 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:40.903195 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.899278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.9.215.3:44057)
0811 18:50:40.899506 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.899510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.899519 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:50:40.899561 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.899564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.899581 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.899861 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:40.899940 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.901092 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902242 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.902789 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902855 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.902923 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.903063 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.903106 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260811 18:50:40.903183 5799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.898972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:44057 (local address 127.0.0.1:60476)
0811 18:50:40.899300 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.899345 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.899419 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.899690 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.899694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.899697 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:40.899783 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.899785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.901232 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.901233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.902152 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.902155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.902869 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.902950 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.902965 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.903034 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.903091 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":106,"threads_started":1}
I20260811 18:50:40.903380 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.899100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.9.215.1:38927)
0811 18:50:40.899278 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.899283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.899510 (+ 227us) server_negotiation.cc:407] Connection header received
0811 18:50:40.899610 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.899613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.899629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.899712 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:40.899924 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.900653 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902106 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.902833 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902902 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.902958 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.903236 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.903286 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260811 18:50:40.903473 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.898972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38927 (local address 127.0.0.1:38994)
0811 18:50:40.899316 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.899350 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.899419 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.899798 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.899799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.899801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:40.899849 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.899851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.900757 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.900759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.902019 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.902021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.902995 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.903060 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.903064 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.903122 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.903167 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260811 18:50:40.903916 5631 tablet_service.cc:1467] Processing CreateTablet for tablet 8a11471866954165aeb1fced538a378b (DEFAULT_TABLE table=test-table [id=c01e5c50e09c40a287b0ebcce010093c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:40.904079 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.898972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:38119 (local address 127.0.0.1:34550)
0811 18:50:40.899586 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:40.899595 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:40.899634 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:40.899919 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:40.899919 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:40.899921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:40.899953 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.899955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.901200 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.901201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.902271 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:40.902273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.903159 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.903160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:40.903211 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.903214 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:50:40.903244 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 18:50:40.904008 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":494,"thread_start_us":116,"threads_started":1}
I20260811 18:50:40.904326 5631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a11471866954165aeb1fced538a378b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:40.904328 5144 tablet_service.cc:1467] Processing CreateTablet for tablet 8a11471866954165aeb1fced538a378b (DEFAULT_TABLE table=test-table [id=c01e5c50e09c40a287b0ebcce010093c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:40.904623 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:40.899123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.9.215.2:38119)
0811 18:50:40.899752 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:40.899755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:40.899766 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:50:40.899799 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:40.899801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:40.899816 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:40.899918 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:40.900005 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.901086 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.902357 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:40.903066 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:40.904326 (+ 1260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:40.904449 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:40.904477 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 18:50:40.904529 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":93,"threads_started":1}
I20260811 18:50:40.904685 5144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a11471866954165aeb1fced538a378b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:40.905290 5392 tablet_service.cc:1467] Processing CreateTablet for tablet 8a11471866954165aeb1fced538a378b (DEFAULT_TABLE table=test-table [id=c01e5c50e09c40a287b0ebcce010093c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:40.905642 5392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a11471866954165aeb1fced538a378b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:40.907522 5811 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Bootstrap starting.
I20260811 18:50:40.907799 5813 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Bootstrap starting.
I20260811 18:50:40.908434 5811 tablet_bootstrap.cc:654] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:40.908478 5813 tablet_bootstrap.cc:654] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:40.908939 5811 log.cc:824] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:40.909046 5813 log.cc:824] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:40.909227 5817 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Bootstrap starting.
I20260811 18:50:40.909780 5813 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: No bootstrap required, opened a new log
I20260811 18:50:40.909794 5817 tablet_bootstrap.cc:654] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:40.909847 5813 ts_tablet_manager.cc:1397] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:40.910281 5817 log.cc:824] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:40.910995 5817 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: No bootstrap required, opened a new log
I20260811 18:50:40.911051 5817 ts_tablet_manager.cc:1397] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:50:40.911589 5813 raft_consensus.cc:348] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:40.911759 5813 raft_consensus.cc:374] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:40.911788 5811 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: No bootstrap required, opened a new log
I20260811 18:50:40.911803 5813 raft_consensus.cc:729] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7445029c454defa4e58fe47fb172e0, State: Initialized, Role: FOLLOWER
I20260811 18:50:40.911854 5811 ts_tablet_manager.cc:1397] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 18:50:40.911919 5813 consensus_queue.cc:260] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:40.912214 5813 ts_tablet_manager.cc:1428] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:50:40.912211 5764 heartbeater.cc:503] Master 127.9.215.62:43843 was elected leader, sending a full tablet report...
I20260811 18:50:40.913228 5817 raft_consensus.cc:348] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:40.913420 5817 raft_consensus.cc:374] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:40.913456 5817 raft_consensus.cc:729] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db4627535f84ab28d28d6e3f76630e8, State: Initialized, Role: FOLLOWER
I20260811 18:50:40.913578 5817 consensus_queue.cc:260] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:40.913885 5817 ts_tablet_manager.cc:1428] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:40.913888 5520 heartbeater.cc:503] Master 127.9.215.62:43843 was elected leader, sending a full tablet report...
I20260811 18:50:40.913916 5811 raft_consensus.cc:348] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:40.914058 5811 raft_consensus.cc:374] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:40.914093 5811 raft_consensus.cc:729] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8972bec699c940d4b10f4099542ea1a7, State: Initialized, Role: FOLLOWER
I20260811 18:50:40.914218 5811 consensus_queue.cc:260] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:40.914505 5811 ts_tablet_manager.cc:1428] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:40.914522 5260 heartbeater.cc:503] Master 127.9.215.62:43843 was elected leader, sending a full tablet report...
W20260811 18:50:40.958540 5521 tablet.cc:2369] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:50:41.046123 5263 tablet.cc:2369] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:41.117408 5826 raft_consensus.cc:482] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:41.117547 5826 raft_consensus.cc:504] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.117909 5826 leader_election.cc:288] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927)
W20260811 18:50:41.120076 5769 tablet.cc:2369] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:41.122002 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.118207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:52599 (local address 127.9.215.1:38927)
0811 18:50:41.118257 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.118259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.118427 (+ 168us) server_negotiation.cc:407] Connection header received
0811 18:50:41.118501 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.118502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.118513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.118555 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:41.118686 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.119376 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.120926 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.121550 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.121614 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.121627 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.121875 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.121930 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:50:41.122007 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.118173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:53897 (local address 127.9.215.3:44057)
0811 18:50:41.118217 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.118219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.118252 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:50:41.118355 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.118356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.118368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.118421 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:41.118616 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.119361 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.120913 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.121525 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.121614 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.121628 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.121876 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.121950 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:50:41.122012 5523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.118115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:44057 (local address 127.9.215.2:53897)
0811 18:50:41.118178 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.118191 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.118271 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.118471 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.118472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.118473 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:41.118543 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.118546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.119488 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.119489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.120825 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.120829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.121673 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.121674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.121770 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.121772 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.121840 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.121890 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":79}
I20260811 18:50:41.122020 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.118115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38927 (local address 127.9.215.2:52599)
0811 18:50:41.118361 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.118369 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.118424 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.118567 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.118567 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.118569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:41.118629 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.118630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.119488 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.119489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.120826 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.120829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.121673 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.121674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.121765 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.121771 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.121840 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.121890 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":137,"thread_start_us":98,"threads_started":1}
I20260811 18:50:41.122416 5670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a11471866954165aeb1fced538a378b" candidate_uuid: "7db4627535f84ab28d28d6e3f76630e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7445029c454defa4e58fe47fb172e0" is_pre_election: true
I20260811 18:50:41.122488 5182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a11471866954165aeb1fced538a378b" candidate_uuid: "7db4627535f84ab28d28d6e3f76630e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8972bec699c940d4b10f4099542ea1a7" is_pre_election: true
I20260811 18:50:41.122614 5182 raft_consensus.cc:2454] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7db4627535f84ab28d28d6e3f76630e8 in term 0.
I20260811 18:50:41.122622 5670 raft_consensus.cc:2454] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7db4627535f84ab28d28d6e3f76630e8 in term 0.
I20260811 18:50:41.122860 5302 leader_election.cc:302] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7db4627535f84ab28d28d6e3f76630e8, 8972bec699c940d4b10f4099542ea1a7; no voters:
I20260811 18:50:41.123090 5826 raft_consensus.cc:2781] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:41.123147 5826 raft_consensus.cc:482] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:41.123170 5826 raft_consensus.cc:3037] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:41.123960 5826 raft_consensus.cc:504] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.124109 5826 leader_election.cc:288] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927)
I20260811 18:50:41.124274 5670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a11471866954165aeb1fced538a378b" candidate_uuid: "7db4627535f84ab28d28d6e3f76630e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7445029c454defa4e58fe47fb172e0"
I20260811 18:50:41.124305 5182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a11471866954165aeb1fced538a378b" candidate_uuid: "7db4627535f84ab28d28d6e3f76630e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8972bec699c940d4b10f4099542ea1a7"
I20260811 18:50:41.124370 5182 raft_consensus.cc:3037] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:41.124370 5670 raft_consensus.cc:3037] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:41.125304 5670 raft_consensus.cc:2454] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7db4627535f84ab28d28d6e3f76630e8 in term 1.
I20260811 18:50:41.125304 5182 raft_consensus.cc:2454] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7db4627535f84ab28d28d6e3f76630e8 in term 1.
I20260811 18:50:41.125504 5302 leader_election.cc:302] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7db4627535f84ab28d28d6e3f76630e8, 8972bec699c940d4b10f4099542ea1a7; no voters:
I20260811 18:50:41.125623 5826 raft_consensus.cc:2781] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:41.125767 5826 raft_consensus.cc:686] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 LEADER]: Becoming Leader. State: Replica: 7db4627535f84ab28d28d6e3f76630e8, State: Running, Role: LEADER
I20260811 18:50:41.125892 5826 consensus_queue.cc:237] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.126716 4896 catalog_manager.cc:5495] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2). New cstate: current_term: 1 leader_uuid: "7db4627535f84ab28d28d6e3f76630e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:41.187857 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.185294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34566 (local address 127.9.215.2:38119)
0811 18:50:41.185369 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.185371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.185379 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:50:41.185422 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.185422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.185446 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.185511 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:41.185684 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.186224 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.187038 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.187242 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.187300 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.187495 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:41.187614 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:41.187706 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.187723 (+ 17us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.187746 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:50:41.189788 5182 raft_consensus.cc:1261] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Refusing update from remote peer 7db4627535f84ab28d28d6e3f76630e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:41.189788 5670 raft_consensus.cc:1261] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Refusing update from remote peer 7db4627535f84ab28d28d6e3f76630e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:41.190138 5826 consensus_queue.cc:1035] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:41.190297 5833 consensus_queue.cc:1035] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:41.192812 5836 mvcc.cc:205] Tried to move back new op lower bound from 7317398491910676480 to 7317398491652145152. Current Snapshot: MvccSnapshot[applied={T|T < 7317398491910676480}]
I20260811 18:50:41.193646 5838 mvcc.cc:205] Tried to move back new op lower bound from 7317398491910676480 to 7317398491652145152. Current Snapshot: MvccSnapshot[applied={T|T < 7317398491910676480 or (T in {7317398491910676480})}]
I20260811 18:50:41.193799 5837 mvcc.cc:205] Tried to move back new op lower bound from 7317398491910676480 to 7317398491652145152. Current Snapshot: MvccSnapshot[applied={T|T < 7317398491910676480}]
I20260811 18:50:41.195333 4895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:41.195425 4895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:41.196249 4895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Sending DeleteTablet for 3 replicas of tablet 8a11471866954165aeb1fced538a378b
I20260811 18:50:41.196759 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 5026
I20260811 18:50:41.196812 5631 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a11471866954165aeb1fced538a378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:60476
I20260811 18:50:41.196833 5144 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a11471866954165aeb1fced538a378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:38994
I20260811 18:50:41.197041 5392 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a11471866954165aeb1fced538a378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:34550
I20260811 18:50:41.212699 5852 tablet_replica.cc:330] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: stopping tablet replica
I20260811 18:50:41.212898 5852 raft_consensus.cc:2229] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:41.213035 5852 raft_consensus.cc:2258] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.213784 5852 ts_tablet_manager.cc:1905] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:41.215528 5852 ts_tablet_manager.cc:1918] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:41.215621 5852 log.cc:1196] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/8a11471866954165aeb1fced538a378b
I20260811 18:50:41.215906 5852 ts_tablet_manager.cc:1939] T 8a11471866954165aeb1fced538a378b P 3b7445029c454defa4e58fe47fb172e0: Deleting consensus metadata
I20260811 18:50:41.216391 4862 catalog_manager.cc:4841] TS 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): tablet 8a11471866954165aeb1fced538a378b (table test-table [id=c01e5c50e09c40a287b0ebcce010093c]) successfully deleted
I20260811 18:50:41.219702 5854 tablet_replica.cc:330] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: stopping tablet replica
I20260811 18:50:41.219879 5854 raft_consensus.cc:2229] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:41.220021 5854 raft_consensus.cc:2258] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.220510 5854 ts_tablet_manager.cc:1905] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:50:41.221194 4863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:38927 (error 108)
I20260811 18:50:41.221467 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 5268
I20260811 18:50:41.221786 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.221650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38927 (local address 127.0.0.1:39008)
0811 18:50:41.221706 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.221750 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:38927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 18:50:41.224817 4863 catalog_manager.cc:4584] TS 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a11471866954165aeb1fced538a378b: Network error: Client connection negotiation failed: client connection to 127.9.215.1:38927: connect: Connection refused (error 111)
I20260811 18:50:41.232487 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 5528
I20260811 18:50:41.232620 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.232480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:38119 (local address 127.0.0.1:34580)
0811 18:50:41.232541 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.232572 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.2:38119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 18:50:41.244553 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 4829
I20260811 18:50:41.265911 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43843
--webserver_interface=127.9.215.62
--webserver_port=33795
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:43843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:41.351423 5857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:41.351676 5857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:41.351699 5857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:41.352926 5857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:41.352977 5857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:41.352991 5857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:41.353004 5857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:41.353029 5857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:41.354627 5857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:43843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:41.354899 5857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:41.355147 5857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:41.364907 5880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.364880 5874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.364880 5868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:41.365006 5857 server_base.cc:1034] running on GCE node
I20260811 18:50:41.365505 5857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:41.365842 5857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:41.366990 5857 hybrid_clock.cc:648] HybridClock initialized: now 1786474241366978 us; error 28 us; skew 500 ppm
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:41.369704 5857 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:41.370246 5857 webserver.cc:460] Webserver started at http://127.9.215.62:33795/ using document root <none> and password file <none>
I20260811 18:50:41.370489 5857 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:41.370559 5857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:41.372071 5857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:50:41.372846 5889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.373028 5857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.373138 5857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c119efe934de493c816f3787ee917074"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "f01614ed02b798e2d51cc021e0714792"
server_key_iv: "9493d1929a5877c4d5590a8cda9a5241"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.373561 5857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:41.400616 5857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:41.403010 5857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:41.403180 5857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:41.407413 5857 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:43843
I20260811 18:50:41.407488 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:43843 every 8 connection(s)
I20260811 18:50:41.407883 5857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:41.409689 6001 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:50:41.410388 6001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap starting.
I20260811 18:50:41.412304 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 5857
I20260811 18:50:41.412588 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:38927
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:43843
--webserver_port=45515
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:41.420185 6001 log.cc:824] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:41.422192 6001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:41.422510 6001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap complete.
I20260811 18:50:41.424896 6001 raft_consensus.cc:348] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:41.425141 6001 raft_consensus.cc:729] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c119efe934de493c816f3787ee917074, State: Initialized, Role: FOLLOWER
I20260811 18:50:41.425295 6001 consensus_queue.cc:260] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:41.425362 6001 raft_consensus.cc:388] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:41.425398 6001 raft_consensus.cc:482] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:41.425449 6001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:50:41.426542 6001 raft_consensus.cc:504] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:41.426659 6001 leader_election.cc:302] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c119efe934de493c816f3787ee917074; no voters:
I20260811 18:50:41.426849 6001 leader_election.cc:288] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:50:41.426906 6009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:50:41.427030 6009 raft_consensus.cc:686] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 LEADER]: Becoming Leader. State: Replica: c119efe934de493c816f3787ee917074, State: Running, Role: LEADER
I20260811 18:50:41.427127 6009 consensus_queue.cc:237] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:41.427155 6001 sys_catalog.cc:564] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:41.427476 6009 sys_catalog.cc:455] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c119efe934de493c816f3787ee917074. Latest consensus state: current_term: 2 leader_uuid: "c119efe934de493c816f3787ee917074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } } }
I20260811 18:50:41.427560 6009 sys_catalog.cc:458] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:41.427480 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c119efe934de493c816f3787ee917074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } } }
I20260811 18:50:41.427757 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:41.427767 6013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:41.428740 6013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:41.428905 6013 catalog_manager.cc:1259] Loaded cluster ID: 2699fb79251241878f017c698dda819a
I20260811 18:50:41.429162 6013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:50:41.429457 6013 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 18:50:41.434042 6013 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 18:50:41.437453 6013 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:41.437709 6013 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Loaded TSK: 0
I20260811 18:50:41.437947 6013 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:50:41.500092 6005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:41.500337 6005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:41.500373 6005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:41.501662 6005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:41.501714 6005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:41.501755 6005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:50:41.503410 6005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:38927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=45515
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:41.503727 6005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:41.503988 6005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:41.510319 6048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.510254 6042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:41.510476 6005 server_base.cc:1034] running on GCE node
W20260811 18:50:41.510290 6041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:41.510767 6005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:41.511027 6005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:41.512193 6005 hybrid_clock.cc:648] HybridClock initialized: now 1786474241512169 us; error 40 us; skew 500 ppm
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:41.515023 6005 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:50:41.515604 6005 webserver.cc:460] Webserver started at http://127.9.215.1:45515/ using document root <none> and password file <none>
I20260811 18:50:41.515846 6005 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:41.515914 6005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:41.517323 6005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.518029 6066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.518203 6005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.518299 6005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "8972bec699c940d4b10f4099542ea1a7"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "ff39850402e282e300b0effd40a43113"
server_key_iv: "1dc6a5b9801242a025f105b73ae5a19f"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.518664 6005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:41.533123 6005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:41.535326 6005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:41.535522 6005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:41.535789 6005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:41.536270 6079 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:50:41.537776 6005 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:50:41.537829 6005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 18:50:41.537855 6005 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:50:41.538509 6005 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:50:41.538551 6005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:41.538615 6079 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Bootstrap starting.
I20260811 18:50:41.546237 6005 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:38927
I20260811 18:50:41.546415 6293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:38927 every 8 connection(s)
I20260811 18:50:41.546840 6005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:41.549942 6079 log.cc:824] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:41.550060 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 6005
I20260811 18:50:41.550390 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:38119
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:43843
--webserver_port=37823
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:41.553714 6079 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:41.554035 6079 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Bootstrap complete.
I20260811 18:50:41.555294 6079 ts_tablet_manager.cc:1397] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.010s
I20260811 18:50:41.556602 6079 raft_consensus.cc:348] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.556959 6079 raft_consensus.cc:729] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8972bec699c940d4b10f4099542ea1a7, State: Initialized, Role: FOLLOWER
I20260811 18:50:41.557132 6079 consensus_queue.cc:260] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.557440 6079 ts_tablet_manager.cc:1428] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:41.562172 6296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.547617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:34215)
0811 18:50:41.550230 (+ 2613us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.550286 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.550998 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.551674 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.551679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.552033 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:41.552314 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.552322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.553902 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.553903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.555105 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.555108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.555937 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.555938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.556083 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.556719 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:41.556739 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:41.558622 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:41.560486 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.560494 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.560496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.560730 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.561007 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.561008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.561008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.561161 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.561531 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:41.561539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:41.561679 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.561769 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.561961 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2563,"thread_start_us":99,"threads_started":1}
I20260811 18:50:41.562172 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.547728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:34215 (local address 127.9.215.62:43843)
0811 18:50:41.547956 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.547969 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.550437 (+ 2468us) server_negotiation.cc:407] Connection header received
0811 18:50:41.551250 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.551258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.551333 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.551517 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:41.552461 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.553770 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.555213 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.555820 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.555929 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.558738 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:41.558742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:41.558743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:41.558754 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:41.560394 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.560814 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.560817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.560818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.560931 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.561228 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.561229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.561229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.561466 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:41.561547 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.561832 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.561960 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":97,"threads_started":1}
I20260811 18:50:41.563226 6294 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43843
I20260811 18:50:41.563333 6294 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:41.563594 6294 heartbeater.cc:511] Master 127.9.215.62:43843 requested a full tablet report, sending...
I20260811 18:50:41.564319 5932 ts_manager.cc:194] Registered new tserver with Master: 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927)
I20260811 18:50:41.565433 5932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:34215
I20260811 18:50:41.566423 6294 heartbeater.cc:503] Master 127.9.215.62:43843 was elected leader, sending a full tablet report...
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.1@KRBTEST.COM
I20260811 18:50:41.573701 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.565194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.9.215.1:38927)
0811 18:50:41.565485 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.565490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.565502 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:50:41.565660 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.565662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.565679 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.565767 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:41.566388 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.567524 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.568323 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.568785 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.568850 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.570969 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:41.570979 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:41.570979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:41.570985 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:41.572482 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.572927 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.572930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.572930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.572990 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.573167 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.573168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.573168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.573310 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:41.573368 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.573544 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.573628 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":71,"threads_started":1}
I20260811 18:50:41.573761 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.564922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38927 (local address 127.0.0.1:39018)
0811 18:50:41.565356 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.565390 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.565456 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.565898 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.565903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.566235 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:41.566298 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.566302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.567597 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.567598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.568251 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.568253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.568847 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.568848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.568903 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.569376 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:41.569381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:41.570879 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:41.572556 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.572563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.572565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.572844 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.573043 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.573044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.573044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.573111 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.573363 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:41.573370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:41.573430 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.573515 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.573643 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":101,"threads_started":1}
I20260811 18:50:41.574213 6169 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a11471866954165aeb1fced538a378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:39018
I20260811 18:50:41.574848 6308 tablet_replica.cc:330] stopping tablet replica
I20260811 18:50:41.574951 6308 raft_consensus.cc:2229] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:41.575023 6308 raft_consensus.cc:2258] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.575315 6308 ts_tablet_manager.cc:1905] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:41.577253 6308 ts_tablet_manager.cc:1918] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:41.577319 6308 log.cc:1196] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/8a11471866954165aeb1fced538a378b
I20260811 18:50:41.577500 6308 ts_tablet_manager.cc:1939] T 8a11471866954165aeb1fced538a378b P 8972bec699c940d4b10f4099542ea1a7: Deleting consensus metadata
I20260811 18:50:41.578007 5891 catalog_manager.cc:4841] TS 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927): tablet 8a11471866954165aeb1fced538a378b (table test-table [id=c01e5c50e09c40a287b0ebcce010093c]) successfully deleted
W20260811 18:50:41.641001 6302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:41.641220 6302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:41.641247 6302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:41.642437 6302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:41.642483 6302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:41.642511 6302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:50:41.644112 6302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:38119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=37823
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:41.644317 6302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:41.644528 6302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:41.649623 6321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.649626 6317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.649593 6319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:41.649901 6302 server_base.cc:1034] running on GCE node
I20260811 18:50:41.650091 6302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:41.650346 6302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:41.651499 6302 hybrid_clock.cc:648] HybridClock initialized: now 1786474241651488 us; error 32 us; skew 500 ppm
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:41.654049 6302 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:50:41.654537 6302 webserver.cc:460] Webserver started at http://127.9.215.2:37823/ using document root <none> and password file <none>
I20260811 18:50:41.654783 6302 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:41.654852 6302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:41.656286 6302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.656975 6335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.657137 6302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:50:41.657214 6302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "7db4627535f84ab28d28d6e3f76630e8"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "17c0f9efb3cd035901d0f2be352858b4"
server_key_iv: "45013b98b6507c54dd384c2dae91e2de"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.657562 6302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:41.669363 6302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:41.671617 6302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:41.671777 6302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:41.672024 6302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:41.672453 6349 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:50:41.673969 6302 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:50:41.674031 6302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 18:50:41.674073 6302 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:50:41.674767 6302 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:50:41.674799 6302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:41.674906 6349 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Bootstrap starting.
I20260811 18:50:41.676587 6349 log.cc:824] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:41.679880 6349 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:41.680215 6349 tablet_bootstrap.cc:492] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Bootstrap complete.
I20260811 18:50:41.680392 6349 ts_tablet_manager.cc:1397] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:50:41.681118 6302 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:38119
I20260811 18:50:41.681246 6541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:38119 every 8 connection(s)
I20260811 18:50:41.681480 6302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:41.682168 6349 raft_consensus.cc:348] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.682504 6349 raft_consensus.cc:729] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db4627535f84ab28d28d6e3f76630e8, State: Initialized, Role: FOLLOWER
I20260811 18:50:41.682643 6349 consensus_queue.cc:260] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.682907 6349 ts_tablet_manager.cc:1428] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:41.690091 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 6302
I20260811 18:50:41.690425 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:44057
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:43843
--webserver_port=39103
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:41.691614 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.682197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:40251 (local address 127.9.215.62:43843)
0811 18:50:41.682276 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.682278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.682539 (+ 261us) server_negotiation.cc:407] Connection header received
0811 18:50:41.682703 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.682704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.682716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.682785 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:41.683537 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.684073 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.685171 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.685457 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.685518 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.688476 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:41.688478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:41.688478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:41.688486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:41.689861 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.690427 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.690428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.690429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.690502 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.690818 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.690819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.690820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.690978 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:41.691054 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.691386 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.691519 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:50:41.691777 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.682153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:40251)
0811 18:50:41.682356 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.682401 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.682583 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.682894 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.682898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.683204 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:41.683405 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.683410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.684190 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.684191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.685072 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.685075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.685576 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.685581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.685706 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.686235 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:41.686250 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:41.688365 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:41.689954 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.689965 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.689968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.690319 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.690599 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.690600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.690600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.690748 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.691085 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:41.691094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:41.691244 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.691351 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.691537 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260811 18:50:41.692386 6544 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43843
I20260811 18:50:41.692510 6544 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:41.692721 6544 heartbeater.cc:511] Master 127.9.215.62:43843 requested a full tablet report, sending...
I20260811 18:50:41.693360 5932 ts_manager.cc:194] Registered new tserver with Master: 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
I20260811 18:50:41.693921 5932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:40251
I20260811 18:50:41.694832 6544 heartbeater.cc:503] Master 127.9.215.62:43843 was elected leader, sending a full tablet report...
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.2@KRBTEST.COM
I20260811 18:50:41.700873 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.693674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:38119 (local address 127.0.0.1:34596)
0811 18:50:41.693726 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.693735 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.693770 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.694087 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.694088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.694262 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:41.694316 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.694319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.695364 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.695365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.695924 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.695926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.696394 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.696395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.696466 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.696875 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:41.696880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:41.698077 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:41.699821 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.699822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.699823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.699998 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.700192 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.700193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.700193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.700255 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.700536 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:41.700537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:41.700592 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.700655 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.700767 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:50:41.700896 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.693754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.9.215.2:38119)
0811 18:50:41.693926 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.693931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.693941 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:50:41.693978 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.693980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.693994 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.694071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:41.694389 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.695238 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.695995 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.696325 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.696385 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.698155 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:41.698164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:41.698164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:41.698171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:41.699741 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.700070 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.700079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.700079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.700134 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.700306 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.700306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.700307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.700470 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:41.700546 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.700695 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.700788 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260811 18:50:41.701408 6429 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a11471866954165aeb1fced538a378b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:34596
I20260811 18:50:41.702140 6553 tablet_replica.cc:330] stopping tablet replica
I20260811 18:50:41.702260 6553 raft_consensus.cc:2229] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:41.702337 6553 raft_consensus.cc:2258] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.702701 6553 ts_tablet_manager.cc:1905] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:41.704807 6553 ts_tablet_manager.cc:1918] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:41.704890 6553 log.cc:1196] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/8a11471866954165aeb1fced538a378b
I20260811 18:50:41.705096 6553 ts_tablet_manager.cc:1939] T 8a11471866954165aeb1fced538a378b P 7db4627535f84ab28d28d6e3f76630e8: Deleting consensus metadata
I20260811 18:50:41.705688 5892 catalog_manager.cc:4841] TS 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): tablet 8a11471866954165aeb1fced538a378b (table test-table [id=c01e5c50e09c40a287b0ebcce010093c]) successfully deleted
W20260811 18:50:41.780414 6550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:41.780630 6550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:41.780655 6550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:41.781811 6550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:41.781855 6550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:41.781881 6550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:50:41.783382 6550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:44057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=39103
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:41.783649 6550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:41.783867 6550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:41.788846 6568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.788839 6564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:41.788849 6563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:41.788942 6550 server_base.cc:1034] running on GCE node
I20260811 18:50:41.789237 6550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:41.789475 6550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:41.790619 6550 hybrid_clock.cc:648] HybridClock initialized: now 1786474241790606 us; error 26 us; skew 500 ppm
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:41.793052 6550 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:50:41.793466 6550 webserver.cc:460] Webserver started at http://127.9.215.3:39103/ using document root <none> and password file <none>
I20260811 18:50:41.793654 6550 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:41.793708 6550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:41.795077 6550 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.795789 6581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.795956 6550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.796041 6550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "3b7445029c454defa4e58fe47fb172e0"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "e82fca9dacf7cf5625848f11b02c187a"
server_key_iv: "1f953f46d7a0c84c1a97f2d5ea643e73"
server_key_version: "encryptionkey@0"
I20260811 18:50:41.796409 6550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:41.813612 6550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:41.815707 6550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:41.815832 6550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:41.816031 6550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:41.816336 6550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:41.816373 6550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.816395 6550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:41.816418 6550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:41.822096 6550 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44057
I20260811 18:50:41.822165 6810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44057 every 8 connection(s)
I20260811 18:50:41.822508 6550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:41.827378 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 6550
Aug 11 18:50:41 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:41.832262 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.823262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:36097 (local address 127.9.215.62:43843)
0811 18:50:41.823348 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.823350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.823630 (+ 280us) server_negotiation.cc:407] Connection header received
0811 18:50:41.824157 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.824159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.824177 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.824231 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:41.825208 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.825704 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.826686 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.826876 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.826931 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.829414 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:41.829415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:41.829416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:41.829423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:41.830938 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.831312 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.831313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.831313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.831371 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.831637 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.831638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.831639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.831751 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:41.831806 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.832087 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.832171 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:50:41.832441 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.823238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:36097)
0811 18:50:41.823433 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.823509 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.824060 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.824316 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.824321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.824899 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:41.825112 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.825117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.825789 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.825790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.826601 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.826604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.826948 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.826948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.827072 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.827755 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:41.827764 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:41.829325 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:41.830999 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.831009 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.831011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.831244 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.831424 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:41.831425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:41.831425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:41.831579 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:41.831805 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:41.831813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:41.831957 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.832041 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.832215 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260811 18:50:41.833325 6812 heartbeater.cc:347] Connected to a master server at 127.9.215.62:43843
I20260811 18:50:41.833441 6812 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:41.833626 6812 heartbeater.cc:511] Master 127.9.215.62:43843 requested a full tablet report, sending...
I20260811 18:50:41.834024 5932 ts_manager.cc:194] Registered new tserver with Master: 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057)
I20260811 18:50:41.834388 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.827871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.9.215.62:43843)
0811 18:50:41.828007 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.828009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.828125 (+ 116us) server_negotiation.cc:407] Connection header received
0811 18:50:41.828203 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.828204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.828213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.828305 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:41.828910 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.829441 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.830442 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.830643 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.830704 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.831901 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:41.831902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:41.831902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:41.831910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:41.833278 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.833657 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.833658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.833659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.833713 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:41.833878 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:41.833878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:41.833879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:41.833993 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:41.834161 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.834200 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.834278 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260811 18:50:41.834504 5932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:36097
I20260811 18:50:41.835139 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:41.837647 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.835483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.9.215.62:43843)
0811 18:50:41.835518 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.835520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.835611 (+ 91us) server_negotiation.cc:407] Connection header received
0811 18:50:41.835681 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.835682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.835712 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.835760 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:41.835856 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.836303 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.837015 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.837197 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.837252 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.837419 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:41.837512 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:41.837566 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.837581 (+ 15us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.837598 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 18:50:41.837930 5932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:41.839608 5932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:41.841622 6429 tablet_service.cc:1467] Processing CreateTablet for tablet 3e17f649d0b04aabadd11e74f23b2410 (DEFAULT_TABLE table=test-table [id=99ed39946e3f4e6a82834e8ecc5ce18c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:41.841845 6429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e17f649d0b04aabadd11e74f23b2410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:41.843505 6349 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Bootstrap starting.
I20260811 18:50:41.843683 6169 tablet_service.cc:1467] Processing CreateTablet for tablet 3e17f649d0b04aabadd11e74f23b2410 (DEFAULT_TABLE table=test-table [id=99ed39946e3f4e6a82834e8ecc5ce18c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:41.844125 6349 tablet_bootstrap.cc:654] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:41.844115 6169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e17f649d0b04aabadd11e74f23b2410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:41.844777 6349 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: No bootstrap required, opened a new log
I20260811 18:50:41.844833 6349 ts_tablet_manager.cc:1397] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.845120 6349 raft_consensus.cc:348] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.845211 6349 raft_consensus.cc:374] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:41.845263 6349 raft_consensus.cc:729] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db4627535f84ab28d28d6e3f76630e8, State: Initialized, Role: FOLLOWER
I20260811 18:50:41.845297 6349 consensus_queue.cc:260] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.845383 6349 ts_tablet_manager.cc:1428] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:41.845479 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.841427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:44057 (local address 127.0.0.1:40972)
0811 18:50:41.841474 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.841484 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.841522 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.841880 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.841881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.841883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:41.841928 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.841929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.843126 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.843127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.844090 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.844094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.845189 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.845190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.845260 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.845289 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.845357 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.845408 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 18:50:41.845497 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.841513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.9.215.3:44057)
0811 18:50:41.841713 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.841718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.841729 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:50:41.841768 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.841770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.841786 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.841877 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:41.841999 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.843020 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.844229 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.845098 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.845177 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.845259 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.845377 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.845422 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
I20260811 18:50:41.845602 6079 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Bootstrap starting.
I20260811 18:50:41.845939 6079 tablet_bootstrap.cc:654] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:41.846565 6079 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: No bootstrap required, opened a new log
I20260811 18:50:41.846616 6079 ts_tablet_manager.cc:1397] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.846500 6667 tablet_service.cc:1467] Processing CreateTablet for tablet 3e17f649d0b04aabadd11e74f23b2410 (DEFAULT_TABLE table=test-table [id=99ed39946e3f4e6a82834e8ecc5ce18c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:41.846879 6667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e17f649d0b04aabadd11e74f23b2410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:41.846949 6079 raft_consensus.cc:348] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.847010 6079 raft_consensus.cc:374] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:41.847038 6079 raft_consensus.cc:729] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8972bec699c940d4b10f4099542ea1a7, State: Initialized, Role: FOLLOWER
I20260811 18:50:41.847076 6079 consensus_queue.cc:260] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.847154 6079 ts_tablet_manager.cc:1428] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:41.849565 6832 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Bootstrap starting.
I20260811 18:50:41.850152 6832 tablet_bootstrap.cc:654] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:41.850606 6832 log.cc:824] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:41.851377 6832 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: No bootstrap required, opened a new log
I20260811 18:50:41.851431 6832 ts_tablet_manager.cc:1397] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:50:41.852768 6832 raft_consensus.cc:348] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.852897 6832 raft_consensus.cc:374] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:41.852918 6832 raft_consensus.cc:729] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7445029c454defa4e58fe47fb172e0, State: Initialized, Role: FOLLOWER
I20260811 18:50:41.853025 6832 consensus_queue.cc:260] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.853318 6832 ts_tablet_manager.cc:1428] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:50:41.853399 6812 heartbeater.cc:503] Master 127.9.215.62:43843 was elected leader, sending a full tablet report...
I20260811 18:50:41.885412 6836 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:41.885558 6836 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.886044 6836 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119), 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927)
I20260811 18:50:41.890321 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.886349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:33881 (local address 127.9.215.1:38927)
0811 18:50:41.886400 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.886403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.886499 (+ 96us) server_negotiation.cc:407] Connection header received
0811 18:50:41.886542 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.886543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.886571 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.886633 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:41.886805 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.887650 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.889223 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.889883 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.889965 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.890059 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.890182 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.890228 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:50:41.890329 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.886362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:50119 (local address 127.9.215.2:38119)
0811 18:50:41.886428 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.886430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.886439 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:50:41.886467 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.886468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.886495 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.886619 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:41.886804 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.887682 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.889222 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.889893 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.889965 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.890059 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.890181 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.890228 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:50:41.890352 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.886229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:38119 (local address 127.9.215.3:50119)
0811 18:50:41.886292 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.886305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.886352 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.886632 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.886634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.886635 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:41.886731 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.886733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.887749 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.887750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.889114 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.889115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.889999 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.890000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.890095 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.890102 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.890168 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.890227 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":74}
I20260811 18:50:41.890362 6837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.886239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38927 (local address 127.9.215.3:33881)
0811 18:50:41.886434 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.886442 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:41.886500 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:41.886638 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:41.886638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:41.886639 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:41.886739 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.886740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.887743 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.887745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.889111 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:41.889114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.889999 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.890000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:41.890095 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.890102 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:50:41.890171 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:50:41.890221 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":73,"thread_start_us":113,"threads_started":1}
I20260811 18:50:41.890719 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e17f649d0b04aabadd11e74f23b2410" candidate_uuid: "3b7445029c454defa4e58fe47fb172e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db4627535f84ab28d28d6e3f76630e8" is_pre_election: true
I20260811 18:50:41.890901 6462 raft_consensus.cc:2454] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b7445029c454defa4e58fe47fb172e0 in term 0.
I20260811 18:50:41.891067 6208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e17f649d0b04aabadd11e74f23b2410" candidate_uuid: "3b7445029c454defa4e58fe47fb172e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8972bec699c940d4b10f4099542ea1a7" is_pre_election: true
I20260811 18:50:41.891211 6208 raft_consensus.cc:2454] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b7445029c454defa4e58fe47fb172e0 in term 0.
I20260811 18:50:41.891187 6584 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8; no voters:
I20260811 18:50:41.891316 6836 raft_consensus.cc:2781] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:41.891378 6836 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:41.891402 6836 raft_consensus.cc:3037] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:41.892143 6836 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.892282 6836 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [CANDIDATE]: Term 1 election: Requested vote from peers 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119), 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927)
I20260811 18:50:41.892444 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e17f649d0b04aabadd11e74f23b2410" candidate_uuid: "3b7445029c454defa4e58fe47fb172e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db4627535f84ab28d28d6e3f76630e8"
I20260811 18:50:41.892534 6462 raft_consensus.cc:3037] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:41.892542 6208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e17f649d0b04aabadd11e74f23b2410" candidate_uuid: "3b7445029c454defa4e58fe47fb172e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8972bec699c940d4b10f4099542ea1a7"
I20260811 18:50:41.892660 6208 raft_consensus.cc:3037] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:41.893256 6462 raft_consensus.cc:2454] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7445029c454defa4e58fe47fb172e0 in term 1.
I20260811 18:50:41.893373 6208 raft_consensus.cc:2454] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7445029c454defa4e58fe47fb172e0 in term 1.
I20260811 18:50:41.893503 6584 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8; no voters:
I20260811 18:50:41.893597 6836 raft_consensus.cc:2781] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:41.893808 6836 raft_consensus.cc:686] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 1 LEADER]: Becoming Leader. State: Replica: 3b7445029c454defa4e58fe47fb172e0, State: Running, Role: LEADER
I20260811 18:50:41.893918 6836 consensus_queue.cc:237] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:41.894825 5932 catalog_manager.cc:5495] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3). New cstate: current_term: 1 leader_uuid: "3b7445029c454defa4e58fe47fb172e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:41.904166 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.901866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.9.215.3:44057)
0811 18:50:41.901911 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:41.901913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:41.901920 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:50:41.901950 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:41.901951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:41.901969 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:41.902029 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:41.902147 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.902603 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.903338 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:41.903608 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:41.903672 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:41.903841 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:41.903940 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:41.903994 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:41.904043 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 18:50:41.904060 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:50:41.905956 6208 raft_consensus.cc:1261] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Refusing update from remote peer 3b7445029c454defa4e58fe47fb172e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:41.905975 6462 raft_consensus.cc:1261] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Refusing update from remote peer 3b7445029c454defa4e58fe47fb172e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:41.906298 6839 consensus_queue.cc:1035] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:41.906420 6836 consensus_queue.cc:1035] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:41.908489 6852 mvcc.cc:205] Tried to move back new op lower bound from 7317398494844157952 to 7317398494798249984. Current Snapshot: MvccSnapshot[applied={T|T < 7317398494844157952}]
I20260811 18:50:41.908535 6856 mvcc.cc:205] Tried to move back new op lower bound from 7317398494844157952 to 7317398494798249984. Current Snapshot: MvccSnapshot[applied={T|T < 7317398494844157952}]
I20260811 18:50:41.909065 6857 mvcc.cc:205] Tried to move back new op lower bound from 7317398494844157952 to 7317398494798249984. Current Snapshot: MvccSnapshot[applied={T|T < 7317398494844157952}]
I20260811 18:50:41.910614 5932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:41.910712 5932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:41.911518 5932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Sending DeleteTablet for 3 replicas of tablet 3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:41.911841 6169 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e17f649d0b04aabadd11e74f23b2410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:39018
I20260811 18:50:41.911875 6429 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e17f649d0b04aabadd11e74f23b2410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:34596
I20260811 18:50:41.911857 6667 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e17f649d0b04aabadd11e74f23b2410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:41 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:40972
I20260811 18:50:41.911936 6308 tablet_replica.cc:330] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: stopping tablet replica
I20260811 18:50:41.911958 6553 tablet_replica.cc:330] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: stopping tablet replica
I20260811 18:50:41.911981 6308 raft_consensus.cc:2229] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:41.912005 6553 raft_consensus.cc:2229] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:41.912034 6308 raft_consensus.cc:2258] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.912052 6553 raft_consensus.cc:2258] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.912200 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 6005
I20260811 18:50:41.925352 6553 ts_tablet_manager.cc:1905] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:41.929912 6553 ts_tablet_manager.cc:1918] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:41.929984 6553 log.cc:1196] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:41.930163 6553 ts_tablet_manager.cc:1939] T 3e17f649d0b04aabadd11e74f23b2410 P 7db4627535f84ab28d28d6e3f76630e8: Deleting consensus metadata
I20260811 18:50:41.932850 5892 catalog_manager.cc:4841] TS 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): tablet 3e17f649d0b04aabadd11e74f23b2410 (table test-table [id=99ed39946e3f4e6a82834e8ecc5ce18c]) successfully deleted
I20260811 18:50:41.954377 6877 tablet_replica.cc:330] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: stopping tablet replica
I20260811 18:50:41.954581 6877 raft_consensus.cc:2229] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:41.954763 6877 raft_consensus.cc:2258] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:41.955335 6877 ts_tablet_manager.cc:1905] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:41.957150 6877 ts_tablet_manager.cc:1918] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:41.957247 6877 log.cc:1196] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:41.957566 6877 ts_tablet_manager.cc:1939] T 3e17f649d0b04aabadd11e74f23b2410 P 3b7445029c454defa4e58fe47fb172e0: Deleting consensus metadata
I20260811 18:50:41.958190 5890 catalog_manager.cc:4841] TS 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): tablet 3e17f649d0b04aabadd11e74f23b2410 (table test-table [id=99ed39946e3f4e6a82834e8ecc5ce18c]) successfully deleted
W20260811 18:50:41.958274 5891 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:38927 (error 108)
I20260811 18:50:41.958525 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 6302
I20260811 18:50:41.970646 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:41.968134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:38927 (local address 127.0.0.1:33312)
0811 18:50:41.970496 (+ 2362us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:41.970590 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:38927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2323}
W20260811 18:50:41.970836 5891 catalog_manager.cc:4584] TS 8972bec699c940d4b10f4099542ea1a7 (127.9.215.1:38927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e17f649d0b04aabadd11e74f23b2410: Network error: Client connection negotiation failed: client connection to 127.9.215.1:38927: connect: Connection refused (error 111)
I20260811 18:50:41.978786 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 6550
I20260811 18:50:41.999306 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 5857
I20260811 18:50:42.020138 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43843
--webserver_interface=127.9.215.62
--webserver_port=33795
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:43843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:42.104967 6880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.105193 6880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.105217 6880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.106434 6880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:42.106487 6880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.106501 6880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.106513 6880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:42.106523 6880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:42.108143 6880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:43843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:43843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=33795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:42.108388 6880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.108636 6880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.113816 6900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.113816 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.113996 6880 server_base.cc:1034] running on GCE node
W20260811 18:50:42.113847 6890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.114334 6880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.114683 6880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.115844 6880 hybrid_clock.cc:648] HybridClock initialized: now 1786474242115823 us; error 37 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.118392 6880 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:42.118873 6880 webserver.cc:460] Webserver started at http://127.9.215.62:33795/ using document root <none> and password file <none>
I20260811 18:50:42.119107 6880 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.119158 6880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.120602 6880 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.121392 6915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.121608 6880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.121711 6880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c119efe934de493c816f3787ee917074"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "f01614ed02b798e2d51cc021e0714792"
server_key_iv: "9493d1929a5877c4d5590a8cda9a5241"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.122089 6880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.132687 6880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.134841 6880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.134990 6880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.139219 6880 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:43843
I20260811 18:50:42.139266 7017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:43843 every 8 connection(s)
I20260811 18:50:42.139683 6880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:42.141399 7018 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:50:42.142030 7018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap starting.
I20260811 18:50:42.145980 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 6880
I20260811 18:50:42.146302 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:38927
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:43843
--webserver_port=45515
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:42.151592 7018 log.cc:824] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:42.153874 7018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:42.154181 7018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074: Bootstrap complete.
I20260811 18:50:42.156545 7018 raft_consensus.cc:348] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:42.156791 7018 raft_consensus.cc:729] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c119efe934de493c816f3787ee917074, State: Initialized, Role: FOLLOWER
I20260811 18:50:42.156968 7018 consensus_queue.cc:260] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:42.157050 7018 raft_consensus.cc:388] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:42.157101 7018 raft_consensus.cc:482] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:42.157150 7018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:50:42.158229 7018 raft_consensus.cc:504] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:42.158341 7018 leader_election.cc:302] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c119efe934de493c816f3787ee917074; no voters:
I20260811 18:50:42.158524 7018 leader_election.cc:288] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:50:42.158603 7024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:50:42.158776 7024 raft_consensus.cc:686] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [term 3 LEADER]: Becoming Leader. State: Replica: c119efe934de493c816f3787ee917074, State: Running, Role: LEADER
I20260811 18:50:42.158815 7018 sys_catalog.cc:564] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:42.158886 7024 consensus_queue.cc:237] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } }
I20260811 18:50:42.159185 7024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c119efe934de493c816f3787ee917074. Latest consensus state: current_term: 3 leader_uuid: "c119efe934de493c816f3787ee917074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } } }
I20260811 18:50:42.159256 7024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:42.159394 7025 sys_catalog.cc:455] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c119efe934de493c816f3787ee917074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c119efe934de493c816f3787ee917074" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 43843 } } }
I20260811 18:50:42.159531 7034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:42.159610 7025 sys_catalog.cc:458] T 00000000000000000000000000000000 P c119efe934de493c816f3787ee917074 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:42.160475 7034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:42.160578 7034 catalog_manager.cc:1259] Loaded cluster ID: 2699fb79251241878f017c698dda819a
I20260811 18:50:42.160612 7034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 18:50:42.160928 7034 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f48b7a149ad google::LogMessage::Fail() at ??:0
@ 0x7f48b7a1788a google::LogMessage::SendToLog() at ??:0
@ 0x7f48b7a144de google::LogMessage::Flush() at ??:0
@ 0x7f48b7a15a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f48bf9350f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f48bf93e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f48b98e20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f48b98d7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f48ba73b6db start_thread at ??:0
@ 0x7f48b651261f clone at ??:0
W20260811 18:50:42.243723 7021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.243939 7021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.243974 7021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.245381 7021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.245441 7021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.245482 7021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:50:42.247210 7021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:38927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=45515
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:42.247509 7021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.247782 7021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.253242 7086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.253289 7085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.253254 7089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.253846 7021 server_base.cc:1034] running on GCE node
I20260811 18:50:42.254091 7021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.254427 7021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.255587 7021 hybrid_clock.cc:648] HybridClock initialized: now 1786474242255575 us; error 31 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.258334 7021 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:50:42.258989 7021 webserver.cc:460] Webserver started at http://127.9.215.1:45515/ using document root <none> and password file <none>
I20260811 18:50:42.259231 7021 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.259284 7021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.260878 7021 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.261677 7103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.261979 7021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.262085 7021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "8972bec699c940d4b10f4099542ea1a7"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "ff39850402e282e300b0effd40a43113"
server_key_iv: "1dc6a5b9801242a025f105b73ae5a19f"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.262504 7021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.283293 7021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.286057 7021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.286217 7021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.286476 7021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:42.286960 7118 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:50:42.288972 7021 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:50:42.289032 7021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 18:50:42.289070 7021 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:50:42.289737 7021 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:50:42.289772 7021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:42.289865 7118 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Bootstrap starting.
I20260811 18:50:42.296883 7021 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:38927
I20260811 18:50:42.297039 7347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:38927 every 8 connection(s)
I20260811 18:50:42.297335 7021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:50:42.301698 7118 log.cc:824] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:42.302690 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 7021
I20260811 18:50:42.302977 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:38119
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:43843
--webserver_port=37823
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:50:42.304816 7118 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:50:42.305176 7118 tablet_bootstrap.cc:492] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Bootstrap complete.
I20260811 18:50:42.306491 7118 ts_tablet_manager.cc:1397] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.007s
I20260811 18:50:42.308051 7118 raft_consensus.cc:348] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:42.308409 7118 raft_consensus.cc:729] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8972bec699c940d4b10f4099542ea1a7, State: Initialized, Role: FOLLOWER
I20260811 18:50:42.308589 7118 consensus_queue.cc:260] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:42.308892 7118 ts_tablet_manager.cc:1428] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260811 18:50:42.368378 7350 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:50:42.297994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:59877)
0811 18:50:42.298172 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.298220 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:42.298727 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:42.368176 (+ 69449us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
W20260811 18:50:42.368652 7348 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:43843 (0 consecutive failures): Network error: Failed to ping master at 127.9.215.62:43843: Client connection negotiation failed: client connection to 127.9.215.62:43843: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:50:42.369014 7350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.368902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:36657)
0811 18:50:42.368950 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.368991 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 18:50:42.369356 7350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.369277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:36335)
0811 18:50:42.369317 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.369340 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 18:50:42.369462 7348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:50:42.390921 7352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.391163 7352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.391192 7352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.392513 7352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.392566 7352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.392594 7352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:50:42.394140 7352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:38119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=37823
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:42.394353 7352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.394560 7352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.399822 7369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.399845 7374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.399842 7371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.400043 7352 server_base.cc:1034] running on GCE node
I20260811 18:50:42.400297 7352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.400550 7352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.401708 7352 hybrid_clock.cc:648] HybridClock initialized: now 1786474242401697 us; error 33 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.404332 7352 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:50:42.404793 7352 webserver.cc:460] Webserver started at http://127.9.215.2:37823/ using document root <none> and password file <none>
I20260811 18:50:42.405021 7352 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.405090 7352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.406423 7352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.407006 7393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.407166 7352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:50:42.407262 7352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "7db4627535f84ab28d28d6e3f76630e8"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "17c0f9efb3cd035901d0f2be352858b4"
server_key_iv: "45013b98b6507c54dd384c2dae91e2de"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.407652 7352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.419082 7352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.421311 7352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.421459 7352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.421698 7352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:42.422000 7352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:42.422041 7352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.422075 7352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:42.422101 7352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.427734 7352 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:38119
I20260811 18:50:42.427815 7621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:38119 every 8 connection(s)
I20260811 18:50:42.428146 7352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:42.429440 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.428860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:60017)
0811 18:50:42.429019 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.429115 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
W20260811 18:50:42.429649 7622 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:43843 (0 consecutive failures): Network error: Failed to ping master at 127.9.215.62:43843: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
I20260811 18:50:42.429944 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.429861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:52925)
0811 18:50:42.429901 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.429927 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 18:50:42.430171 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.430122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:52891)
0811 18:50:42.430142 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.430160 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260811 18:50:42.430250 7622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:50:42.434051 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 7352
I20260811 18:50:42.434288 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:44057
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:43843
--webserver_port=39103
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:42.518914 7631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:42.519146 7631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:42.519173 7631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:42.520438 7631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:42.520491 7631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:42.520517 7631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:50:42.522048 7631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:44057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=39103
--tserver_master_addrs=127.9.215.62:43843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:42.522310 7631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:42.522610 7631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:42.527897 7647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:42.527940 7642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.527983 7631 server_base.cc:1034] running on GCE node
W20260811 18:50:42.527916 7643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:42.528348 7631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:42.528602 7631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:42.529771 7631 hybrid_clock.cc:648] HybridClock initialized: now 1786474242529748 us; error 34 us; skew 500 ppm
Aug 11 18:50:42 dist-test-slave-pww0 krb5kdc[4796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:42.532548 7631 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:50:42.533058 7631 webserver.cc:460] Webserver started at http://127.9.215.3:39103/ using document root <none> and password file <none>
I20260811 18:50:42.533308 7631 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:42.533361 7631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:42.534752 7631 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.535431 7673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.535672 7631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:50:42.535753 7631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "3b7445029c454defa4e58fe47fb172e0"
format_stamp: "Formatted at 2026-08-11 18:50:40 on dist-test-slave-pww0"
server_key: "e82fca9dacf7cf5625848f11b02c187a"
server_key_iv: "1f953f46d7a0c84c1a97f2d5ea643e73"
server_key_version: "encryptionkey@0"
I20260811 18:50:42.536104 7631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:42.549216 7631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:42.551522 7631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:42.551656 7631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:42.551880 7631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:42.552165 7631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:42.552197 7631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.552219 7631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:42.552235 7631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:42.557999 7631 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:44057
I20260811 18:50:42.558079 7898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:44057 every 8 connection(s)
I20260811 18:50:42.558409 7631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:42.559569 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.559149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:53889)
0811 18:50:42.559299 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.559382 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260811 18:50:42.559957 7899 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:43843 (0 consecutive failures): Network error: Failed to ping master at 127.9.215.62:43843: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
I20260811 18:50:42.560307 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.560213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:54443)
0811 18:50:42.560258 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.560287 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:50:42.560307 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 7631
I20260811 18:50:42.560619 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:42.560533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:48137)
0811 18:50:42.560572 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:42.560596 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 18:50:42.560767 7899 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:50:43.370172 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:43.369875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:49391)
0811 18:50:43.370023 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:43.370079 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260811 18:50:43.430882 7908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:43.430645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:43817)
0811 18:50:43.430783 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:43.430825 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260811 18:50:43.561439 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:43.561187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:40335)
0811 18:50:43.561331 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:43.561378 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260811 18:50:44.005371 7910 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:44.005505 7910 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:44.005883 7910 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
Aug 11 18:50:44 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.3@KRBTEST.COM
Aug 11 18:50:44 dist-test-slave-pww0 krb5kdc[4796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.2@KRBTEST.COM
I20260811 18:50:44.018700 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.006207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:41841 (local address 127.9.215.2:38119)
0811 18:50:44.006385 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:44.006398 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:44.006426 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:50:44.007041 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:44.007045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:44.007108 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:44.007275 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:44.008415 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.009872 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.011515 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.011993 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.012137 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:44.015170 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:44.015174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:44.015174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:44.015186 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:44.017075 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017484 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:44.017488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:44.017488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:44.017605 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017850 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:44.017851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:44.017852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:44.018131 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:44.018213 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:44.018431 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 18:50:44.018582 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260811 18:50:44.018698 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.006291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:39891 (local address 127.9.215.3:44057)
0811 18:50:44.006426 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:44.006433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:44.006462 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:50:44.007042 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:44.007045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:44.007108 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:44.007275 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:44.008420 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.009873 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.011540 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.012001 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.012111 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:44.014867 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:44.014873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:44.014873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:44.014888 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:44.016717 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017236 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:44.017241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:44.017241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:44.017373 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017727 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:44.017729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:44.017729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:44.018040 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:44.018104 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:44.018437 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 18:50:44.018578 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":51,"threads_started":1}
I20260811 18:50:44.018699 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.006090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:44057 (local address 127.9.215.1:39891)
0811 18:50:44.006261 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:44.006274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:44.006338 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:44.007378 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:44.007382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:44.007955 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:44.008246 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:44.008255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.010046 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.010047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:44.011366 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:44.011371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.012105 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.012113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:44.012240 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:44.012890 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:44.012900 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:44.014712 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:44.016811 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:44.016821 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:44.016824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:44.017129 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:44.017474 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:44.017475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:44.017660 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:44.018109 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:44.018117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:44.018271 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 18:50:44.018364 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:50:44.018600 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":80,"thread_start_us":100,"threads_started":1}
I20260811 18:50:44.018808 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.006090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:38119 (local address 127.9.215.1:41841)
0811 18:50:44.006277 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:44.006285 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:44.006338 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:44.007378 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:44.007382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:44.007955 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:44.008246 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:44.008255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.010056 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.010057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:44.011366 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:44.011371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:44.012114 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:44.012115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:44.012241 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:44.012912 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:44.012916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:44.015041 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:44.017167 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:44.017168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:44.017384 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:44.017706 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:44.017707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:44.017707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:44.017780 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:44.018202 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:44.018203 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:44.018270 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 18:50:44.018364 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:50:44.018684 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":180,"spinlock_wait_cycles":15616,"thread_start_us":121,"threads_started":1}
W20260811 18:50:44.019573 7104 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
W20260811 18:50:44.020006 7108 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:44.020054 7108 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8972bec699c940d4b10f4099542ea1a7; no voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8
I20260811 18:50:44.020207 7910 raft_consensus.cc:2726] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:50:44.371016 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.370827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:35651)
0811 18:50:44.370899 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:44.370961 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 18:50:44.431646 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.431365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:32835)
0811 18:50:44.431534 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:44.431587 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":63,"threads_started":1}
I20260811 18:50:44.562251 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:44.561976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:33819)
0811 18:50:44.562150 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:44.562203 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
I20260811 18:50:45.371893 7920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:45.371650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:40865)
0811 18:50:45.371785 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:45.371835 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260811 18:50:45.432468 7921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:45.432188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:37777)
0811 18:50:45.432368 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:45.432420 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260811 18:50:45.562953 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:45.562715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:34783)
0811 18:50:45.562869 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:45.562905 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260811 18:50:45.846319 7926 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:45.846423 7926 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:45.846602 7926 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
W20260811 18:50:45.847052 7104 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
W20260811 18:50:45.847198 7108 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:45.847278 7108 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8972bec699c940d4b10f4099542ea1a7; no voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8
I20260811 18:50:45.847345 7926 raft_consensus.cc:2726] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:50:46.372678 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:46.372392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:54861)
0811 18:50:46.372581 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:46.372628 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260811 18:50:46.433137 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:46.432938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:40439)
0811 18:50:46.433057 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:46.433094 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260811 18:50:46.563748 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:46.563498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:54191)
0811 18:50:46.563635 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:46.563686 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260811 18:50:47.373608 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:47.373299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:44409)
0811 18:50:47.373457 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:47.373530 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260811 18:50:47.433938 7940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:47.433702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:34933)
0811 18:50:47.433837 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:47.433887 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260811 18:50:47.564553 7942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:47.564310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:46141)
0811 18:50:47.564447 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:47.564487 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260811 18:50:48.133912 7945 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:48.134044 7945 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:48.134194 7945 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
W20260811 18:50:48.134555 7104 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
W20260811 18:50:48.134670 7108 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:48.134733 7108 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8972bec699c940d4b10f4099542ea1a7; no voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8
I20260811 18:50:48.134804 7945 raft_consensus.cc:2726] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:50:48.374517 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:48.374273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:45653)
0811 18:50:48.374418 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:48.374464 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260811 18:50:48.434705 7949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:48.434428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:39643)
0811 18:50:48.434593 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:48.434633 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260811 18:50:48.565292 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:48.565023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:53525)
0811 18:50:48.565200 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:48.565244 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260811 18:50:49.375341 7953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:49.375088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:45271)
0811 18:50:49.375245 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:49.375292 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260811 18:50:49.435559 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:49.435244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:47371)
0811 18:50:49.435423 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:49.435507 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
I20260811 18:50:49.566185 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:49.565903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:58919)
0811 18:50:49.566072 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:49.566122 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260811 18:50:50.216657 7960 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:50.216764 7960 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:50.216917 7960 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
W20260811 18:50:50.217353 7108 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
W20260811 18:50:50.217456 7104 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:50.217500 7104 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8972bec699c940d4b10f4099542ea1a7; no voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8
I20260811 18:50:50.217602 7960 raft_consensus.cc:2726] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:50:50.376189 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:50.375923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:47931)
0811 18:50:50.376079 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:50.376131 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260811 18:50:50.436484 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:50.436173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:53787)
0811 18:50:50.436378 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:50.436436 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":117,"threads_started":1}
I20260811 18:50:50.567044 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:50.566780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:43343)
0811 18:50:50.566941 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:50.566990 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260811 18:50:51.377069 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:51.376780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:41927)
0811 18:50:51.376966 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:51.377017 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260811 18:50:51.437170 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:51.436935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:54205)
0811 18:50:51.437083 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:51.437119 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260811 18:50:51.567863 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:51.567625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:49461)
0811 18:50:51.567766 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:51.567813 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260811 18:50:52.377923 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:52.377691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:38869)
0811 18:50:52.377826 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:52.377873 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260811 18:50:52.437959 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:52.437730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:36771)
0811 18:50:52.437857 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:52.437905 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260811 18:50:52.568642 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:52.568374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:36469)
0811 18:50:52.568531 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:52.568591 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260811 18:50:53.378762 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:53.378490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:36785)
0811 18:50:53.378644 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:53.378693 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260811 18:50:53.397356 7977 raft_consensus.cc:482] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:53.397461 7977 raft_consensus.cc:504] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7445029c454defa4e58fe47fb172e0" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 44057 } } peers { permanent_uuid: "7db4627535f84ab28d28d6e3f76630e8" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38119 } } peers { permanent_uuid: "8972bec699c940d4b10f4099542ea1a7" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 38927 } }
I20260811 18:50:53.397611 7977 leader_election.cc:288] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057), 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119)
W20260811 18:50:53.398145 7108 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7db4627535f84ab28d28d6e3f76630e8 (127.9.215.2:38119): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
W20260811 18:50:53.398283 7104 leader_election.cc:341] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b7445029c454defa4e58fe47fb172e0 (127.9.215.3:44057): Not found: Tablet not found: 3e17f649d0b04aabadd11e74f23b2410
I20260811 18:50:53.398355 7104 leader_election.cc:302] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8972bec699c940d4b10f4099542ea1a7; no voters: 3b7445029c454defa4e58fe47fb172e0, 7db4627535f84ab28d28d6e3f76630e8
I20260811 18:50:53.398429 7977 raft_consensus.cc:2726] T 3e17f649d0b04aabadd11e74f23b2410 P 8972bec699c940d4b10f4099542ea1a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:50:53.438738 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:53.438482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:45281)
0811 18:50:53.438637 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:53.438688 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260811 18:50:53.569422 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:53.569167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:39377)
0811 18:50:53.569321 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:53.569371 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260811 18:50:54.379534 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:54.379297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:46039)
0811 18:50:54.379420 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:54.379476 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260811 18:50:54.439448 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:54.439218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:57401)
0811 18:50:54.439350 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:54.439397 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260811 18:50:54.570158 7989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:54.569899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:44811)
0811 18:50:54.570061 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:54.570104 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260811 18:50:55.380306 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:55.380057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:48513)
0811 18:50:55.380191 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:55.380243 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260811 18:50:55.440248 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:55.439984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:53919)
0811 18:50:55.440135 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:55.440179 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260811 18:50:55.570968 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:55.570706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:57243)
0811 18:50:55.570868 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:55.570918 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260811 18:50:56.381096 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:56.380808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:60945)
0811 18:50:56.380978 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:56.381030 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260811 18:50:56.441037 8006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:56.440780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:34965)
0811 18:50:56.440936 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:56.440986 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260811 18:50:56.571857 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:56.571626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:51241)
0811 18:50:56.571758 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:56.571806 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260811 18:50:57.381901 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:57.381644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.1:42937)
0811 18:50:57.381808 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:57.381850 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260811 18:50:57.441926 8012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:57.441635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.2:60115)
0811 18:50:57.441830 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:57.441875 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":124,"threads_started":1}
I20260811 18:50:57.560935 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 7021
I20260811 18:50:57.581610 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:57.577711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:43843 (local address 127.9.215.3:33449)
0811 18:50:57.581471 (+ 3760us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:57.581541 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:43843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3731,"thread_start_us":2141,"threads_started":1}
I20260811 18:50:57.585664 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 7352
I20260811 18:50:57.598557 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 7631
2026-08-11T18:50:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18792 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:50:57 dist-test-slave-pww0 krb5kdc[8017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:50:57 dist-test-slave-pww0 krb5kdc[8017](info): set up 2 sockets
Aug 11 18:50:57 dist-test-slave-pww0 krb5kdc[8017](info): commencing operation
krb5kdc: starting...
W20260811 18:50:58.829912 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:50:58 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:50:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:50:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:50:58.967811 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40635
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:40635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:59.053906 8049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:59.054143 8049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:59.054179 8049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:59.055729 8049 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:50:59.055794 8049 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:50:59.055810 8049 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:50:59.055825 8049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:50:59.055841 8049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:59.055859 8049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:59.055883 8049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:50:59.055907 8049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:50:59.057617 8049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:40635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:59.057933 8049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:59.058176 8049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:59.064069 8059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.064086 8058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.064222 8062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:59.064076 8049 server_base.cc:1034] running on GCE node
I20260811 18:50:59.064564 8049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:59.064908 8049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:59.066072 8049 hybrid_clock.cc:648] HybridClock initialized: now 1786474259066038 us; error 55 us; skew 500 ppm
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:59.072436 8049 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:50:59.072995 8049 webserver.cc:460] Webserver started at http://127.9.215.62:45551/ using document root <none> and password file <none>
I20260811 18:50:59.073222 8049 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:59.073266 8049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:59.073365 8049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:59.074472 8049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "135c0193b15846e5a66ecf8a47f43c29"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "bae51f786b0e7ed6474dfafcc589338c"
server_key_iv: "32be1e534e61b4914589acdab47fb9c0"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.074788 8049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "135c0193b15846e5a66ecf8a47f43c29"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "bae51f786b0e7ed6474dfafcc589338c"
server_key_iv: "32be1e534e61b4914589acdab47fb9c0"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.076246 8049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:50:59.077039 8073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.077201 8049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:50:59.077304 8049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "135c0193b15846e5a66ecf8a47f43c29"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "bae51f786b0e7ed6474dfafcc589338c"
server_key_iv: "32be1e534e61b4914589acdab47fb9c0"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.077371 8049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:59.089057 8049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:59.091219 8049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:59.091346 8049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:59.095508 8049 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:40635
I20260811 18:50:59.095613 8175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:40635 every 8 connection(s)
I20260811 18:50:59.095949 8049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:50:59.096951 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:59.099397 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap starting.
I20260811 18:50:59.100059 8178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:59.100340 8178 log.cc:824] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:59.101087 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: No bootstrap required, opened a new log
I20260811 18:50:59.102767 8178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:50:59.102919 8178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:59.102953 8178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135c0193b15846e5a66ecf8a47f43c29, State: Initialized, Role: FOLLOWER
I20260811 18:50:59.103097 8178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:50:59.103163 8178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:50:59.103195 8178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:50:59.103245 8178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:59.103902 8178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:50:59.104004 8178 leader_election.cc:302] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135c0193b15846e5a66ecf8a47f43c29; no voters:
I20260811 18:50:59.104022 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 8049
I20260811 18:50:59.104120 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:50:59.104192 8178 leader_election.cc:288] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:50:59.104256 8181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:59.104350 10076 external_mini_cluster.cc:1428] Setting key 90cf3552412454fc6d67d0d6efa319a6
I20260811 18:50:59.104435 8181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 LEADER]: Becoming Leader. State: Replica: 135c0193b15846e5a66ecf8a47f43c29, State: Running, Role: LEADER
I20260811 18:50:59.104473 8178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:50:59.104526 8181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:50:59.105562 8185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 135c0193b15846e5a66ecf8a47f43c29. Latest consensus state: current_term: 1 leader_uuid: "135c0193b15846e5a66ecf8a47f43c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } } }
I20260811 18:50:59.105576 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "135c0193b15846e5a66ecf8a47f43c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } } }
I20260811 18:50:59.105770 8185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: This master's current role is: LEADER
I20260811 18:50:59.105798 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: This master's current role is: LEADER
W20260811 18:50:59.106787 8214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:50:59.106848 8214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:50:59.106905 8204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:50:59.107096 8204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:50:59.108629 8204 catalog_manager.cc:1347] Generated new cluster ID: 74e13e3a5940461f99dd808f429bd8dc
I20260811 18:50:59.108680 8204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:59.117373 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.104912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.9.215.62:40635)
0811 18:50:59.106133 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.106142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.106172 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:50:59.106302 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.106309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.106375 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.106568 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:59.107367 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.108808 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.109881 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.110565 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.110686 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.113542 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:59.113547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:59.113547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:59.113564 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:59.115581 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.115997 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.116001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.116001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.116115 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.116321 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.116321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.116322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.116648 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:59.116785 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.116955 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.117108 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":80,"threads_started":1}
I20260811 18:50:59.129642 8204 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:50:59.130514 8204 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:50:59.137662 8204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Generated new TSK 0
I20260811 18:50:59.137883 8204 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:50:59.168655 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40635
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:59.257870 8228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:59.258119 8228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:59.258157 8228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:59.259495 8228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:59.259559 8228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:59.259629 8228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:50:59.261368 8228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:59.261638 8228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:59.262012 8228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:59.267777 8241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.267802 8242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:59.267974 8228 server_base.cc:1034] running on GCE node
W20260811 18:50:59.267793 8247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:59.268472 8228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:59.268769 8228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:59.269949 8228 hybrid_clock.cc:648] HybridClock initialized: now 1786474259269935 us; error 31 us; skew 500 ppm
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:59.272832 8228 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:50:59.273360 8228 webserver.cc:460] Webserver started at http://127.9.215.1:32921/ using document root <none> and password file <none>
I20260811 18:50:59.273597 8228 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:59.273653 8228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:59.273777 8228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:59.274919 8228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "9eb6a3dde24d9b0d1bed09e8e1c731c5"
server_key_iv: "29165c827efa26b096af074601bba7a5"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.275300 8228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "9eb6a3dde24d9b0d1bed09e8e1c731c5"
server_key_iv: "29165c827efa26b096af074601bba7a5"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.276796 8228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:50:59.277674 8258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.277871 8228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:50:59.277967 8228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "9eb6a3dde24d9b0d1bed09e8e1c731c5"
server_key_iv: "29165c827efa26b096af074601bba7a5"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.278036 8228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:59.286240 8228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:59.288691 8228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:59.288820 8228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:59.289032 8228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:59.289327 8228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:59.289369 8228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.289392 8228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:59.289409 8228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.295246 8228 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:41131
I20260811 18:50:59.295308 8484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:41131 every 8 connection(s)
I20260811 18:50:59.295714 8228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:59.305546 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 8228
I20260811 18:50:59.305796 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:50:59.306012 10076 external_mini_cluster.cc:1428] Setting key b49c89f7c867b12731c723c2cbed1bef
I20260811 18:50:59.306723 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.296546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:34321 (local address 127.9.215.62:40635)
0811 18:50:59.296643 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.296645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.296836 (+ 191us) server_negotiation.cc:407] Connection header received
0811 18:50:59.297380 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.297382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.297398 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.297530 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:59.298582 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.299199 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.300433 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.300652 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.300717 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.303299 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:59.303301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:59.303302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:59.303310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:59.304808 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.305294 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.305296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.305296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.305473 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.305935 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.305936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.305936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.306068 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:59.306125 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.306516 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.306634 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:50:59.306908 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.296443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:34321)
0811 18:50:59.296703 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.296747 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.297268 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.297624 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.297628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.298247 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:59.298468 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.298474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.299331 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.299332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.300326 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.300329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.300728 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.300728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.300843 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.301440 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:59.301451 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:59.303101 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:59.304913 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:59.304925 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:59.304928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:59.305198 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:59.305666 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:59.305667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:59.305668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:59.305858 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:59.306141 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:59.306151 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:59.306344 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.306461 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.306653 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":147,"threads_started":1}
I20260811 18:50:59.307446 8487 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40635
I20260811 18:50:59.307585 8487 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:59.307796 8487 heartbeater.cc:511] Master 127.9.215.62:40635 requested a full tablet report, sending...
I20260811 18:50:59.308337 8099 ts_manager.cc:194] Registered new tserver with Master: d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131)
I20260811 18:50:59.308974 8099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:34321
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:50:59.334928 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40635
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:59.421638 8496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:59.421878 8496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:59.421905 8496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:59.423105 8496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:59.423146 8496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:59.423174 8496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:50:59.424842 8496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:59.425086 8496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:59.425292 8496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:59.430423 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.430609 8506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.430701 8508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:59.430886 8496 server_base.cc:1034] running on GCE node
I20260811 18:50:59.431102 8496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:59.431393 8496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:59.432569 8496 hybrid_clock.cc:648] HybridClock initialized: now 1786474259432553 us; error 31 us; skew 500 ppm
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:59.435204 8496 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:50:59.435741 8496 webserver.cc:460] Webserver started at http://127.9.215.2:33413/ using document root <none> and password file <none>
I20260811 18:50:59.435984 8496 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:59.436040 8496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:59.436175 8496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:59.437278 8496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "5418ff2b56e54987b23758470949167c"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "3e4714204b06231201f168637b81079f"
server_key_iv: "da0274f03e6f4830e5f68157412963a1"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.437633 8496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "5418ff2b56e54987b23758470949167c"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "3e4714204b06231201f168637b81079f"
server_key_iv: "da0274f03e6f4830e5f68157412963a1"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.439074 8496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:50:59.439913 8518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.440089 8496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:50:59.440194 8496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "5418ff2b56e54987b23758470949167c"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "3e4714204b06231201f168637b81079f"
server_key_iv: "da0274f03e6f4830e5f68157412963a1"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.440274 8496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:59.449779 8496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:59.452124 8496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:59.452291 8496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:59.452535 8496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:59.452888 8496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:59.452925 8496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.452956 8496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:59.452973 8496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.459069 8496 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:39043
I20260811 18:50:59.459138 8733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:39043 every 8 connection(s)
I20260811 18:50:59.459537 8496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:50:59.461320 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 8496
I20260811 18:50:59.461438 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:50:59.461659 10076 external_mini_cluster.cc:1428] Setting key 146d3e0a612c09382bdb424951ab2db5
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:59.470674 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.460339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:52651 (local address 127.9.215.62:40635)
0811 18:50:59.460406 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.460407 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.460644 (+ 237us) server_negotiation.cc:407] Connection header received
0811 18:50:59.461162 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.461163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.461176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.461236 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:59.462485 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.463005 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.464592 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.464790 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.464852 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.467606 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:59.467607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:59.467608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:59.467617 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:59.468979 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.469543 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.469544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.469544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.469628 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.469913 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.469913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.469913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.470021 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:59.470066 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.470482 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.470599 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:50:59.470963 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.460308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:52651)
0811 18:50:59.460489 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.460537 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.461067 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.461334 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.461338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.462111 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:59.462384 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.462391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.463121 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.463122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.464488 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.464492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.464867 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.464868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.465008 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.465629 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:59.465642 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:59.467427 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:59.469079 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:59.469090 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:59.469093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:59.469442 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:59.469690 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:59.469691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:59.469692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:59.469851 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:59.470076 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:59.470083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:59.470298 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.470429 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.470623 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
I20260811 18:50:59.471498 8734 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40635
I20260811 18:50:59.471642 8734 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:59.471840 8734 heartbeater.cc:511] Master 127.9.215.62:40635 requested a full tablet report, sending...
I20260811 18:50:59.472229 8099 ts_manager.cc:194] Registered new tserver with Master: 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
I20260811 18:50:59.472657 8099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:52651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:50:59.490792 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40635
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:50:59.577078 8746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:50:59.577334 8746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:50:59.577363 8746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:50:59.578629 8746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:50:59.578671 8746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:50:59.578701 8746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:50:59.580396 8746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:50:59.580648 8746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:50:59.580886 8746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:50:59.586285 8754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.586381 8757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:50:59.586292 8753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:50:59.586637 8746 server_base.cc:1034] running on GCE node
I20260811 18:50:59.586879 8746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:50:59.587149 8746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:50:59.588296 8746 hybrid_clock.cc:648] HybridClock initialized: now 1786474259588284 us; error 35 us; skew 500 ppm
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:50:59.590921 8746 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:50:59.591393 8746 webserver.cc:460] Webserver started at http://127.9.215.3:45541/ using document root <none> and password file <none>
I20260811 18:50:59.591629 8746 fs_manager.cc:359] Metadata directory not provided
I20260811 18:50:59.591682 8746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:50:59.591817 8746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:50:59.592950 8746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "f6b440134bb440bda60c5e0585d97f72"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "a5c2a04c83a55758e416cdcedf803ebb"
server_key_iv: "137c977d78d5a636d858a9b02e4f96ac"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.593310 8746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "f6b440134bb440bda60c5e0585d97f72"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "a5c2a04c83a55758e416cdcedf803ebb"
server_key_iv: "137c977d78d5a636d858a9b02e4f96ac"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.594646 8746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:50:59.595346 8765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.595562 8746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:50:59.595666 8746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "f6b440134bb440bda60c5e0585d97f72"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "a5c2a04c83a55758e416cdcedf803ebb"
server_key_iv: "137c977d78d5a636d858a9b02e4f96ac"
server_key_version: "encryptionkey@0"
I20260811 18:50:59.595732 8746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:50:59.612190 8746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:50:59.614459 8746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:50:59.614612 8746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:50:59.614864 8746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:50:59.615195 8746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:50:59.615231 8746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.615260 8746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:50:59.615285 8746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:50:59.621342 8746 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:36845
I20260811 18:50:59.621428 9005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:36845 every 8 connection(s)
I20260811 18:50:59.621734 8746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:50:59.627512 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 8746
I20260811 18:50:59.627609 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:50:59.627827 10076 external_mini_cluster.cc:1428] Setting key 8fe88a66a98f7d72ce3ce7e4f5aa1491
Aug 11 18:50:59 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474259, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:50:59.631826 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.622510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:46235 (local address 127.9.215.62:40635)
0811 18:50:59.622611 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.622613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.622812 (+ 199us) server_negotiation.cc:407] Connection header received
0811 18:50:59.623334 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.623335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.623347 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.623398 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:59.624441 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.624936 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.625988 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.626193 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.626253 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.628844 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:59.628846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:59.628846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:59.628853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:59.630367 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.630814 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.630815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.630815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.630873 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.631165 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.631166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.631166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.631286 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:59.631337 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.631675 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.631755 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:50:59.631997 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.622433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:46235)
0811 18:50:59.622677 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.622723 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.623232 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.623494 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.623499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.624119 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:50:59.624333 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.624338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.625031 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.625032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.625898 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.625901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.626273 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.626275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.626413 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.627053 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:50:59.627062 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:50:59.628727 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:50:59.630472 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:59.630482 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:59.630484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:59.630746 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:59.630964 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:50:59.630965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:50:59.630965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:50:59.631100 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:50:59.631344 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:50:59.631350 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:50:59.631537 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.631630 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.631788 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
I20260811 18:50:59.632463 9007 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40635
I20260811 18:50:59.632577 9007 heartbeater.cc:464] Registering TS with master...
I20260811 18:50:59.632776 9007 heartbeater.cc:511] Master 127.9.215.62:40635 requested a full tablet report, sending...
I20260811 18:50:59.633147 8099 ts_manager.cc:194] Registered new tserver with Master: f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845)
I20260811 18:50:59.633534 8099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:46235
I20260811 18:50:59.639779 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:50:59.647408 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.641161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.9.215.62:40635)
0811 18:50:59.641253 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.641255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.641295 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:50:59.641349 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.641350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.641360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.641408 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:50:59.641991 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.642486 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.643272 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.643575 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.643638 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.645005 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:50:59.645006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:50:59.645006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:50:59.645014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:50:59.646336 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.646699 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.646699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.646699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.646748 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:50:59.646963 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:50:59.646963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:50:59.646963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:50:59.647107 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:50:59.647179 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.647274 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.647337 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:50:59.648960 8099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:50:59.650583 8099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:50:59.657526 9033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.653055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.0.0.1:34250)
0811 18:50:59.653560 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.653569 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.653625 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.653869 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.653870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.653873 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:59.653906 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.653908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.655296 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.655297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.656194 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.656197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.657156 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.657157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.657257 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.657276 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.657353 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.657417 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":69,"thread_start_us":132,"threads_started":1}
I20260811 18:50:59.657653 9035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.653260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.9.215.3:36845)
0811 18:50:59.653475 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.653480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.653814 (+ 334us) server_negotiation.cc:407] Connection header received
0811 18:50:59.653862 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.653864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.653881 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.653953 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:59.654168 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.654897 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.656486 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.657196 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.657268 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.657316 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.657525 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.657576 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
I20260811 18:50:59.657716 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.653292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.9.215.2:39043)
0811 18:50:59.653496 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.653500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.653511 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:50:59.653556 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.653558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.653573 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.653934 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:59.654013 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.655145 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.656590 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.657286 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.657362 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.657425 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.657586 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.657639 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
I20260811 18:50:59.657672 9027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.653046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:39043 (local address 127.0.0.1:33364)
0811 18:50:59.653241 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.653289 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.653363 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.653749 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.653753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.653755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:59.653838 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.653840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.655260 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.655261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.656436 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.656438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.657368 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.657369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.657435 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.657438 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.657496 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.657534 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":25,"thread_start_us":64,"threads_started":1}
I20260811 18:50:59.657896 9030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.653046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36845 (local address 127.0.0.1:50022)
0811 18:50:59.653705 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.653713 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.653754 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.654035 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.654035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.654037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:59.654095 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.654096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.655001 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.655002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.656384 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.656388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.657353 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.657354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.657425 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.657431 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.657487 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.657534 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":90,"thread_start_us":92,"threads_started":1}
I20260811 18:50:59.658053 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.653362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.9.215.1:41131)
0811 18:50:59.653539 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.653542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.653617 (+ 75us) server_negotiation.cc:407] Connection header received
0811 18:50:59.653689 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.653691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.653707 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.653785 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:59.653980 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.655173 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.656302 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.657061 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.657142 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.657213 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.657919 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.657975 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260811 18:50:59.658684 8868 tablet_service.cc:1467] Processing CreateTablet for tablet 4300843ad0cd404aa41eb21934503629 (DEFAULT_TABLE table=test-table [id=3c26962b033e4d5893433cd2735e5fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:59.658685 8351 tablet_service.cc:1467] Processing CreateTablet for tablet 4300843ad0cd404aa41eb21934503629 (DEFAULT_TABLE table=test-table [id=3c26962b033e4d5893433cd2735e5fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:59.658933 8613 tablet_service.cc:1467] Processing CreateTablet for tablet 4300843ad0cd404aa41eb21934503629 (DEFAULT_TABLE table=test-table [id=3c26962b033e4d5893433cd2735e5fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:50:59.659075 8351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4300843ad0cd404aa41eb21934503629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:59.659077 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4300843ad0cd404aa41eb21934503629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:59.659227 8613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4300843ad0cd404aa41eb21934503629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:50:59.662252 9040 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Bootstrap starting.
I20260811 18:50:59.662528 9042 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Bootstrap starting.
I20260811 18:50:59.662760 9044 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap starting.
I20260811 18:50:59.662995 9040 tablet_bootstrap.cc:654] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:59.663435 9044 tablet_bootstrap.cc:654] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:59.663674 9040 log.cc:824] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:59.664042 9044 log.cc:824] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:59.664301 9042 tablet_bootstrap.cc:654] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Neither blocks nor log segments found. Creating new log.
I20260811 18:50:59.664619 9040 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: No bootstrap required, opened a new log
I20260811 18:50:59.664688 9040 ts_tablet_manager.cc:1397] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:50:59.664831 9044 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: No bootstrap required, opened a new log
I20260811 18:50:59.664918 9044 ts_tablet_manager.cc:1397] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:50:59.664951 9042 log.cc:824] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:50:59.666270 9042 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: No bootstrap required, opened a new log
I20260811 18:50:59.666356 9042 ts_tablet_manager.cc:1397] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:50:59.666512 9040 raft_consensus.cc:348] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.666666 9040 raft_consensus.cc:374] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:59.666700 9040 raft_consensus.cc:729] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b440134bb440bda60c5e0585d97f72, State: Initialized, Role: FOLLOWER
I20260811 18:50:59.666846 9040 consensus_queue.cc:260] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.666786 9044 raft_consensus.cc:348] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.666960 9044 raft_consensus.cc:374] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:59.667006 9044 raft_consensus.cc:729] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b5c236ffcc4bcdaf74270c110481f5, State: Initialized, Role: FOLLOWER
I20260811 18:50:59.667143 9040 ts_tablet_manager.cc:1428] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:50:59.667140 9044 consensus_queue.cc:260] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.667273 9007 heartbeater.cc:503] Master 127.9.215.62:40635 was elected leader, sending a full tablet report...
I20260811 18:50:59.667438 9044 ts_tablet_manager.cc:1428] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:50:59.667557 8487 heartbeater.cc:503] Master 127.9.215.62:40635 was elected leader, sending a full tablet report...
I20260811 18:50:59.668519 9042 raft_consensus.cc:348] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.668700 9042 raft_consensus.cc:374] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:50:59.668735 9042 raft_consensus.cc:729] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5418ff2b56e54987b23758470949167c, State: Initialized, Role: FOLLOWER
I20260811 18:50:59.668820 9042 consensus_queue.cc:260] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.669047 9042 ts_tablet_manager.cc:1428] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:50:59.669100 8734 heartbeater.cc:503] Master 127.9.215.62:40635 was elected leader, sending a full tablet report...
W20260811 18:50:59.710191 8736 tablet.cc:2369] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:50:59.752173 9055 raft_consensus.cc:482] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:50:59.752332 9055 raft_consensus.cc:504] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.752686 9055 leader_election.cc:288] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
I20260811 18:50:59.756830 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.752964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:51385 (local address 127.9.215.1:41131)
0811 18:50:59.753019 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.753021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.753304 (+ 283us) server_negotiation.cc:407] Connection header received
0811 18:50:59.753333 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.753333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.753346 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.753392 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:59.753527 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.754219 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.755793 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.756409 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.756489 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.756501 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.756705 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.756770 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:50:59.756829 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.752964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:51269 (local address 127.9.215.2:39043)
0811 18:50:59.753012 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.753014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.753059 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:50:59.753150 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.753151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.753165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.753220 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:50:59.753401 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.754151 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.755789 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.756407 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.756488 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.756501 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.756705 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.756749 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:50:59.756845 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.752900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:39043 (local address 127.9.215.3:51269)
0811 18:50:59.752997 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.753010 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.753081 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.753245 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.753246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.753248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:59.753331 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.753333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.754272 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.754274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.755677 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.755682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.756513 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.756514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.756623 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.756628 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.756692 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.756744 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":90}
I20260811 18:50:59.756856 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.752904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.9.215.3:51385)
0811 18:50:59.753199 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.753209 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:50:59.753269 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:50:59.753404 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:50:59.753405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:50:59.753417 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:50:59.753468 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.753469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.754298 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.754299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.755677 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:50:59.755682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.756513 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.756514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:50:59.756625 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.756628 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:50:59.756692 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:50:59.756744 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":116,"thread_start_us":129,"threads_started":1}
I20260811 18:50:59.757301 8387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4300843ad0cd404aa41eb21934503629" candidate_uuid: "f6b440134bb440bda60c5e0585d97f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" is_pre_election: true
I20260811 18:50:59.757301 8648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4300843ad0cd404aa41eb21934503629" candidate_uuid: "f6b440134bb440bda60c5e0585d97f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5418ff2b56e54987b23758470949167c" is_pre_election: true
I20260811 18:50:59.757485 8387 raft_consensus.cc:2454] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b440134bb440bda60c5e0585d97f72 in term 0.
I20260811 18:50:59.757485 8648 raft_consensus.cc:2454] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b440134bb440bda60c5e0585d97f72 in term 0.
I20260811 18:50:59.757773 8775 leader_election.cc:302] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5, f6b440134bb440bda60c5e0585d97f72; no voters:
I20260811 18:50:59.757938 9055 raft_consensus.cc:2781] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:50:59.758018 9055 raft_consensus.cc:482] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:50:59.758036 9055 raft_consensus.cc:3037] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:59.758800 9055 raft_consensus.cc:504] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.758944 9055 leader_election.cc:288] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [CANDIDATE]: Term 1 election: Requested vote from peers d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
I20260811 18:50:59.759209 8648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4300843ad0cd404aa41eb21934503629" candidate_uuid: "f6b440134bb440bda60c5e0585d97f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5418ff2b56e54987b23758470949167c"
I20260811 18:50:59.759209 8387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4300843ad0cd404aa41eb21934503629" candidate_uuid: "f6b440134bb440bda60c5e0585d97f72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
I20260811 18:50:59.759323 8648 raft_consensus.cc:3037] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:59.759323 8387 raft_consensus.cc:3037] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:50:59.760304 8648 raft_consensus.cc:2454] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b440134bb440bda60c5e0585d97f72 in term 1.
I20260811 18:50:59.760304 8387 raft_consensus.cc:2454] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b440134bb440bda60c5e0585d97f72 in term 1.
I20260811 18:50:59.760521 8775 leader_election.cc:302] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5, f6b440134bb440bda60c5e0585d97f72; no voters:
I20260811 18:50:59.760622 9055 raft_consensus.cc:2781] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:50:59.760782 9055 raft_consensus.cc:686] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 1 LEADER]: Becoming Leader. State: Replica: f6b440134bb440bda60c5e0585d97f72, State: Running, Role: LEADER
I20260811 18:50:59.760881 9055 consensus_queue.cc:237] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:50:59.761828 8096 catalog_manager.cc:5495] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b440134bb440bda60c5e0585d97f72 (127.9.215.3). New cstate: current_term: 1 leader_uuid: "f6b440134bb440bda60c5e0585d97f72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } health_report { overall_health: UNKNOWN } } }
I20260811 18:50:59.774919 9035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.772664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.9.215.3:36845)
0811 18:50:59.772717 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:50:59.772719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:50:59.772729 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:50:59.772791 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:50:59.772792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:50:59.772813 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:50:59.772874 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:50:59.772979 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.773451 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.774273 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:50:59.774471 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:50:59.774524 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:50:59.774650 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:50:59.774764 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:50:59.774822 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:50:59.774834 (+ 12us) server_negotiation.cc:299] Negotiation successful
0811 18:50:59.774853 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:50:59.776940 8648 raft_consensus.cc:1261] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Refusing update from remote peer f6b440134bb440bda60c5e0585d97f72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:59.776942 8387 raft_consensus.cc:1261] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Refusing update from remote peer f6b440134bb440bda60c5e0585d97f72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:50:59.777274 9055 consensus_queue.cc:1035] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:59.777426 9065 consensus_queue.cc:1035] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:50:59.780205 9075 mvcc.cc:205] Tried to move back new op lower bound from 7317398568043569152 to 7317398567981350912. Current Snapshot: MvccSnapshot[applied={T|T < 7317398568043569152}]
I20260811 18:50:59.780534 9078 mvcc.cc:205] Tried to move back new op lower bound from 7317398568043569152 to 7317398567981350912. Current Snapshot: MvccSnapshot[applied={T|T < 7317398568043569152}]
I20260811 18:50:59.780761 9077 mvcc.cc:205] Tried to move back new op lower bound from 7317398568043569152 to 7317398567981350912. Current Snapshot: MvccSnapshot[applied={T|T < 7317398568043569152}]
I20260811 18:50:59.782804 8096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:59.782919 8096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:50:59.783818 8096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Sending DeleteTablet for 3 replicas of tablet 4300843ad0cd404aa41eb21934503629
I20260811 18:50:59.784123 8868 tablet_service.cc:1514] Processing DeleteTablet for tablet 4300843ad0cd404aa41eb21934503629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:59 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:50022
I20260811 18:50:59.784181 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 8228
I20260811 18:50:59.784222 8613 tablet_service.cc:1514] Processing DeleteTablet for tablet 4300843ad0cd404aa41eb21934503629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:59 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:33364
I20260811 18:50:59.784283 8351 tablet_service.cc:1514] Processing DeleteTablet for tablet 4300843ad0cd404aa41eb21934503629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:59 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:34250
I20260811 18:50:59.805841 9093 tablet_replica.cc:330] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: stopping tablet replica
I20260811 18:50:59.806010 9093 raft_consensus.cc:2229] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:50:59.806155 9093 raft_consensus.cc:2258] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:59.808271 9094 tablet_replica.cc:330] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: stopping tablet replica
I20260811 18:50:59.808439 9094 raft_consensus.cc:2229] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:50:59.808571 9094 raft_consensus.cc:2258] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:50:59.808991 9093 ts_tablet_manager.cc:1905] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:59.810848 9093 ts_tablet_manager.cc:1918] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:59.810954 9093 log.cc:1196] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/4300843ad0cd404aa41eb21934503629
I20260811 18:50:59.811224 9093 ts_tablet_manager.cc:1939] T 4300843ad0cd404aa41eb21934503629 P f6b440134bb440bda60c5e0585d97f72: Deleting consensus metadata
I20260811 18:50:59.811793 8074 catalog_manager.cc:4841] TS f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): tablet 4300843ad0cd404aa41eb21934503629 (table test-table [id=3c26962b033e4d5893433cd2735e5fa5]) successfully deleted
I20260811 18:50:59.812381 9094 ts_tablet_manager.cc:1905] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:50:59.814119 9094 ts_tablet_manager.cc:1918] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:50:59.814316 9094 log.cc:1196] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/4300843ad0cd404aa41eb21934503629
I20260811 18:50:59.814672 9094 ts_tablet_manager.cc:1939] T 4300843ad0cd404aa41eb21934503629 P 5418ff2b56e54987b23758470949167c: Deleting consensus metadata
I20260811 18:50:59.815287 8075 catalog_manager.cc:4841] TS 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): tablet 4300843ad0cd404aa41eb21934503629 (table test-table [id=3c26962b033e4d5893433cd2735e5fa5]) successfully deleted
W20260811 18:50:59.817790 8077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:41131 (error 108)
I20260811 18:50:59.818394 9030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:50:59.818252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.0.0.1:34260)
0811 18:50:59.818311 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 18:50:59.818356 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:41131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 18:50:59.818434 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 8496
W20260811 18:50:59.829777 8077 catalog_manager.cc:4584] TS d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4300843ad0cd404aa41eb21934503629: Network error: Client connection negotiation failed: client connection to 127.9.215.1:41131: connect: Connection refused (error 111)
I20260811 18:50:59.842754 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 8746
I20260811 18:50:59.871786 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 8049
I20260811 18:51:19.888025 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40635
--webserver_interface=127.9.215.62
--webserver_port=45551
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:40635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:19.975056 9098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:19.975302 9098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:19.975327 9098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:19.976636 9098 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:51:19.976692 9098 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:51:19.976706 9098 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:51:19.976715 9098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:51:19.976728 9098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:19.976750 9098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:19.976779 9098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:51:19.976795 9098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:51:19.978471 9098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:40635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:19.978765 9098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:19.979015 9098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:19.989470 9112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:19.989493 9110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:19.989470 9109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:19.989991 9098 server_base.cc:1034] running on GCE node
I20260811 18:51:19.990270 9098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:19.990617 9098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:19.991811 9098 hybrid_clock.cc:648] HybridClock initialized: now 1786474279991777 us; error 50 us; skew 500 ppm
Aug 11 18:51:19 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474279, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:19.998142 9098 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:51:19.998769 9098 webserver.cc:460] Webserver started at http://127.9.215.62:45551/ using document root <none> and password file <none>
I20260811 18:51:19.999024 9098 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:19.999106 9098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:20.000718 9098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:20.001609 9126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.001837 9098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:20.001947 9098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "135c0193b15846e5a66ecf8a47f43c29"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "bae51f786b0e7ed6474dfafcc589338c"
server_key_iv: "32be1e534e61b4914589acdab47fb9c0"
server_key_version: "encryptionkey@0"
I20260811 18:51:20.002406 9098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:20.021229 9098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:20.023437 9098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:20.023634 9098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:20.027988 9098 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:40635
I20260811 18:51:20.028064 9233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:40635 every 8 connection(s)
I20260811 18:51:20.028501 9098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:51:20.030136 9234 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:51:20.030823 9234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap starting.
I20260811 18:51:20.034909 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 9098
I20260811 18:51:20.035298 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:41131
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:40635
--webserver_port=32921
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:51:20.041016 9234 log.cc:824] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:20.043125 9234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:51:20.043503 9234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap complete.
I20260811 18:51:20.045910 9234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:20.046162 9234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135c0193b15846e5a66ecf8a47f43c29, State: Initialized, Role: FOLLOWER
I20260811 18:51:20.046316 9234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:20.046387 9234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:51:20.046442 9234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:51:20.046491 9234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:51:20.047676 9234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:20.047783 9234 leader_election.cc:302] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135c0193b15846e5a66ecf8a47f43c29; no voters:
I20260811 18:51:20.047976 9234 leader_election.cc:288] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:51:20.048048 9238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:51:20.048280 9234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:51:20.048277 9238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 LEADER]: Becoming Leader. State: Replica: 135c0193b15846e5a66ecf8a47f43c29, State: Running, Role: LEADER
I20260811 18:51:20.048487 9238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:20.048806 9246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 135c0193b15846e5a66ecf8a47f43c29. Latest consensus state: current_term: 2 leader_uuid: "135c0193b15846e5a66ecf8a47f43c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } } }
I20260811 18:51:20.048933 9246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:20.049129 9241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "135c0193b15846e5a66ecf8a47f43c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } } }
I20260811 18:51:20.049211 9241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:20.049422 9261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:51:20.050369 9261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:51:20.050513 9261 catalog_manager.cc:1259] Loaded cluster ID: 74e13e3a5940461f99dd808f429bd8dc
I20260811 18:51:20.050585 9261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:51:20.051982 9261 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:51:20.052202 9261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Loaded TSK: 0
I20260811 18:51:20.065950 9261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Generated new TSK 1
I20260811 18:51:20.066094 9261 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:51:20.124349 9236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:20.124606 9236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:20.124648 9236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:20.125949 9236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:20.126000 9236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:20.126029 9236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:51:20.127640 9236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:41131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=32921
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:20.127887 9236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:20.128129 9236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:20.133503 9288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:20.133503 9289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:20.133535 9292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:20.134277 9236 server_base.cc:1034] running on GCE node
I20260811 18:51:20.134521 9236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:20.134799 9236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:20.135982 9236 hybrid_clock.cc:648] HybridClock initialized: now 1786474280135961 us; error 33 us; skew 500 ppm
Aug 11 18:51:20 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474280, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:20.138762 9236 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:51:20.139210 9236 webserver.cc:460] Webserver started at http://127.9.215.1:32921/ using document root <none> and password file <none>
I20260811 18:51:20.139431 9236 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:20.139530 9236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:20.141050 9236 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:20.141783 9303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.141968 9236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:20.142064 9236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "9eb6a3dde24d9b0d1bed09e8e1c731c5"
server_key_iv: "29165c827efa26b096af074601bba7a5"
server_key_version: "encryptionkey@0"
I20260811 18:51:20.142439 9236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:20.150871 9236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:20.153347 9236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:20.153513 9236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:20.153787 9236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:20.154327 9317 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:51:20.155925 9236 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:51:20.155982 9236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 18:51:20.156011 9236 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:51:20.156701 9236 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:51:20.156756 9236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 18:51:20.156817 9317 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap starting.
I20260811 18:51:20.164597 9236 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:41131
I20260811 18:51:20.164776 9543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:41131 every 8 connection(s)
I20260811 18:51:20.165180 9236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:51:20.168943 9317 log.cc:824] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:20.171871 9317 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:51:20.172092 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 9236
I20260811 18:51:20.172199 9317 tablet_bootstrap.cc:492] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap complete.
I20260811 18:51:20.172415 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:39043
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:40635
--webserver_port=33413
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:51:20.173941 9317 ts_tablet_manager.cc:1397] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
Aug 11 18:51:20 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474280, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:20.175673 9317 raft_consensus.cc:348] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:51:20.176343 9317 raft_consensus.cc:729] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b5c236ffcc4bcdaf74270c110481f5, State: Initialized, Role: FOLLOWER
I20260811 18:51:20.176522 9317 consensus_queue.cc:260] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } }
I20260811 18:51:20.176851 9317 ts_tablet_manager.cc:1428] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 18:51:20.179354 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.165995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:60803)
0811 18:51:20.166333 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.166394 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.167106 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.168382 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.168388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.168798 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:20.169076 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.169096 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.170425 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.170426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.171266 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.171270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.172016 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.172017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.172176 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.172966 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:20.172977 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:20.174822 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:20.177342 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:20.177347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:20.177349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:20.177652 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:20.177985 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:20.177986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:20.177987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:20.178129 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:20.178616 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:20.178622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:20.178791 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.178908 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.179121 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":112,"threads_started":1}
I20260811 18:51:20.179382 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.166087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:60803 (local address 127.9.215.62:40635)
0811 18:51:20.167806 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.167818 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.167878 (+ 60us) server_negotiation.cc:407] Connection header received
0811 18:51:20.168052 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.168061 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.168134 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.168318 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:20.169242 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.170289 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.171359 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.171879 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.172535 (+ 656us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.174970 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:20.174976 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:20.174977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:20.174992 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:20.177234 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.177768 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.177772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.177773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.177907 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.178193 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.178193 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.178194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.178530 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:20.178615 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.178992 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.179145 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1652,"thread_start_us":100,"threads_started":1}
I20260811 18:51:20.180609 9548 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40635
I20260811 18:51:20.180727 9548 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:20.180918 9548 heartbeater.cc:511] Master 127.9.215.62:40635 requested a full tablet report, sending...
I20260811 18:51:20.181774 9157 ts_manager.cc:194] Registered new tserver with Master: d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131)
I20260811 18:51:20.183049 9157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:60803
I20260811 18:51:20.184263 9548 heartbeater.cc:503] Master 127.9.215.62:40635 was elected leader, sending a full tablet report...
I20260811 18:51:20.188520 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.182482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.0.0.1:36700)
0811 18:51:20.183760 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.183804 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.183888 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.184241 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.184244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.184246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:20.184318 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.184320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.185866 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.185868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.187168 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.187171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.188180 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.188181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.188268 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.188282 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.188374 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.188436 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1233,"thread_start_us":100,"threads_started":1}
I20260811 18:51:20.188519 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.183387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.9.215.1:41131)
0811 18:51:20.183587 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.183591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.183974 (+ 383us) server_negotiation.cc:407] Connection header received
0811 18:51:20.184016 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.184019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.184090 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.184164 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:20.184520 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.185766 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.187258 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.188087 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.188163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.188224 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.188390 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.188434 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
I20260811 18:51:20.189283 9402 tablet_service.cc:1514] Processing DeleteTablet for tablet 4300843ad0cd404aa41eb21934503629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:50:59 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36700
I20260811 18:51:20.189909 9562 tablet_replica.cc:330] stopping tablet replica
I20260811 18:51:20.190026 9562 raft_consensus.cc:2229] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:20.190099 9562 raft_consensus.cc:2258] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:20.190466 9562 ts_tablet_manager.cc:1905] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:20.193092 9562 ts_tablet_manager.cc:1918] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:20.193193 9562 log.cc:1196] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal/wals/4300843ad0cd404aa41eb21934503629
I20260811 18:51:20.193421 9562 ts_tablet_manager.cc:1939] T 4300843ad0cd404aa41eb21934503629 P d9b5c236ffcc4bcdaf74270c110481f5: Deleting consensus metadata
I20260811 18:51:20.194031 9130 catalog_manager.cc:4841] TS d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131): tablet 4300843ad0cd404aa41eb21934503629 (table test-table [id=3c26962b033e4d5893433cd2735e5fa5]) successfully deleted
W20260811 18:51:20.263512 9557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:20.263748 9557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:20.263774 9557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:20.264990 9557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:20.265038 9557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:20.265065 9557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:51:20.266613 9557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:39043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=33413
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:20.266899 9557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:20.267153 9557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:20.272459 9575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:20.272449 9583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:20.272466 9576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:20.272817 9557 server_base.cc:1034] running on GCE node
I20260811 18:51:20.273043 9557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:20.273276 9557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:20.274430 9557 hybrid_clock.cc:648] HybridClock initialized: now 1786474280274413 us; error 30 us; skew 500 ppm
Aug 11 18:51:20 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474280, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:20.277062 9557 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:51:20.277513 9557 webserver.cc:460] Webserver started at http://127.9.215.2:33413/ using document root <none> and password file <none>
I20260811 18:51:20.277751 9557 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:20.277820 9557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:20.279175 9557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:20.279733 9597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.279886 9557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:51:20.279973 9557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "5418ff2b56e54987b23758470949167c"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "3e4714204b06231201f168637b81079f"
server_key_iv: "da0274f03e6f4830e5f68157412963a1"
server_key_version: "encryptionkey@0"
I20260811 18:51:20.280315 9557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:20.292115 9557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:20.294344 9557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:20.294505 9557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:20.294736 9557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:20.295040 9557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:20.295075 9557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.295106 9557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:20.295133 9557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.301689 9557 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:39043
I20260811 18:51:20.301746 9809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:39043 every 8 connection(s)
I20260811 18:51:20.302114 9557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:51:20 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474280, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:20.309762 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 9557
I20260811 18:51:20.310057 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:36845
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:40635
--webserver_port=45541
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:51:20.312242 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.302889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:33581 (local address 127.9.215.62:40635)
0811 18:51:20.302956 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.302958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.303190 (+ 232us) server_negotiation.cc:407] Connection header received
0811 18:51:20.303784 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.303786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.303800 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.303861 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:20.304869 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.305427 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.306530 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.306790 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.306856 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.309061 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:20.309063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:20.309063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:20.309070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:20.310543 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.311044 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.311045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.311045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.311118 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.311408 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.311409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.311410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.311563 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:20.311621 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.312011 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.312144 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:51:20.312295 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.302887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:33581)
0811 18:51:20.303064 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.303103 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.303654 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.303929 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.303933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.304528 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:20.304758 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.304764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.305510 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.305511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.306430 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.306434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.306876 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.306876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.307001 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.307532 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:20.307550 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:20.308948 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:20.310634 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:20.310644 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:20.310647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:20.310953 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:20.311181 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:20.311182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:20.311182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:20.311334 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:20.311620 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:20.311626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:20.311822 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.311952 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.312111 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260811 18:51:20.313037 9810 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40635
I20260811 18:51:20.313131 9810 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:20.313294 9810 heartbeater.cc:511] Master 127.9.215.62:40635 requested a full tablet report, sending...
I20260811 18:51:20.313716 9157 ts_manager.cc:194] Registered new tserver with Master: 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
I20260811 18:51:20.314203 9157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:33581
W20260811 18:51:20.397132 9814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:20.397362 9814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:20.397387 9814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:20.398615 9814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:20.398670 9814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:20.398697 9814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:51:20.400352 9814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:36845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=45541
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:20.400636 9814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:20.400888 9814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:20.406329 9823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:20.406404 9825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:20.406349 9822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:20.406431 9814 server_base.cc:1034] running on GCE node
I20260811 18:51:20.406806 9814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:20.407073 9814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:20.408214 9814 hybrid_clock.cc:648] HybridClock initialized: now 1786474280408197 us; error 34 us; skew 500 ppm
Aug 11 18:51:20 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474280, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:20.410854 9814 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:51:20.411329 9814 webserver.cc:460] Webserver started at http://127.9.215.3:45541/ using document root <none> and password file <none>
I20260811 18:51:20.411579 9814 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:20.411651 9814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:20.413065 9814 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:20.413637 9838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.413803 9814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:51:20.413895 9814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "f6b440134bb440bda60c5e0585d97f72"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "a5c2a04c83a55758e416cdcedf803ebb"
server_key_iv: "137c977d78d5a636d858a9b02e4f96ac"
server_key_version: "encryptionkey@0"
I20260811 18:51:20.414260 9814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:20.421376 9814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:20.423576 9814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:20.423727 9814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:20.423972 9814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:20.424286 9814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:20.424320 9814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.424350 9814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:20.424374 9814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:20.430162 9814 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:36845
I20260811 18:51:20.430222 10063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:36845 every 8 connection(s)
I20260811 18:51:20.430672 9814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:51:20 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474280, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:20.437187 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 9814
I20260811 18:51:20.440492 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.431442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:34677 (local address 127.9.215.62:40635)
0811 18:51:20.431513 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.431515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.431754 (+ 239us) server_negotiation.cc:407] Connection header received
0811 18:51:20.432260 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.432261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.432274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.432336 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:20.433334 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.433839 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.434884 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.435084 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.435143 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.437436 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:20.437438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:20.437438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:20.437446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:20.439017 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.439476 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.439478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.439479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.439539 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.439798 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.439799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.439799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.439917 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:20.439974 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.440325 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.440423 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:51:20.440717 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.431420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:34677)
0811 18:51:20.431608 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.431647 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.432155 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.432413 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.432418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.433006 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:20.433215 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.433220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.433928 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.433929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.434795 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.434797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.435161 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.435162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.435294 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.435879 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:20.435888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:20.437343 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:20.439107 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:20.439119 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:20.439123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:20.439382 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:20.439607 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:20.439608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:20.439608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:20.439738 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:20.439973 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:20.439982 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:20.440146 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.440253 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.440445 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260811 18:51:20.441350 10064 heartbeater.cc:347] Connected to a master server at 127.9.215.62:40635
I20260811 18:51:20.441473 10064 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:20.441712 10064 heartbeater.cc:511] Master 127.9.215.62:40635 requested a full tablet report, sending...
I20260811 18:51:20.442181 9157 ts_manager.cc:194] Registered new tserver with Master: f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845)
I20260811 18:51:20.442620 9157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:34677
I20260811 18:51:20.444389 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.437635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.9.215.62:40635)
0811 18:51:20.437780 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.437782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.437866 (+ 84us) server_negotiation.cc:407] Connection header received
0811 18:51:20.437926 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.437927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.437941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.438003 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:20.438683 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.439188 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.439991 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.440288 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.440350 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.441803 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:20.441805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:20.441805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:20.441813 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:20.443254 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.443652 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.443653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.443653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.443706 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.443875 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.443876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.443876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.443971 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:20.444138 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.444204 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.444292 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260811 18:51:20.445298 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:51:20.448154 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.445651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.9.215.62:40635)
0811 18:51:20.445727 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.445729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.445856 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:51:20.445925 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.445926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.445960 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.446015 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:51:20.446105 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.446572 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.447365 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.447619 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.447705 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.447876 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:51:20.447929 (+ 53us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 18:51:20.448083 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35464: token expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 18:51:20.448233 10072 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35464: token expired
W20260811 18:51:20.448432 9019 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.215.62:40635)
I20260811 18:51:20.454617 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.448692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.9.215.62:40635)
0811 18:51:20.448805 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.448807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.448814 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:51:20.448832 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.448833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.448842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.448887 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:20.449400 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.449933 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.450622 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.450851 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.450906 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.452120 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:20.452122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:20.452123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:20.452131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:20.453454 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.453816 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.453817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.453818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.453880 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:20.454044 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:20.454045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:20.454045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:20.454220 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:20.454315 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.454426 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.454522 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 18:51:20.456048 9157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:51:20.457778 9157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:51:20.460558 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 058a00d8f9c44ed3811fa43cc53f88dc (DEFAULT_TABLE table=test-table [id=6c220796d04d48a1b54d1fdfe6ee5629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:20.460871 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058a00d8f9c44ed3811fa43cc53f88dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:20.462404 9317 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap starting.
I20260811 18:51:20.462754 9317 tablet_bootstrap.cc:654] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:20.463253 9317 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: No bootstrap required, opened a new log
I20260811 18:51:20.463308 9317 ts_tablet_manager.cc:1397] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:51:20.463425 9317 raft_consensus.cc:348] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.463522 9317 raft_consensus.cc:374] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:20.463574 9317 raft_consensus.cc:729] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b5c236ffcc4bcdaf74270c110481f5, State: Initialized, Role: FOLLOWER
I20260811 18:51:20.463629 9317 consensus_queue.cc:260] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.463717 9317 ts_tablet_manager.cc:1428] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 18:51:20.464445 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.460102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:39043 (local address 127.0.0.1:48276)
0811 18:51:20.460281 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.460291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.460341 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.460762 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.460763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.460765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:20.460799 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.460800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.461985 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.461986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.463106 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.463108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.464177 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.464179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.464257 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.464262 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.464313 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.464363 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
I20260811 18:51:20.464687 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.460260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48276 (local address 127.9.215.2:39043)
0811 18:51:20.460461 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.460466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.460474 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:51:20.460506 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.460508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.460521 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.460599 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:20.460876 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.461891 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.463237 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.464012 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.464085 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.464144 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.464388 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.464432 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
I20260811 18:51:20.464761 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.460022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36845 (local address 127.0.0.1:42364)
0811 18:51:20.460072 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.460084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.460118 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.460611 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.460612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.460615 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:20.460668 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.460670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.461778 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.461779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.462940 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.462942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.464548 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.464550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.464615 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.464619 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.464649 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.464693 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":20}
I20260811 18:51:20.465107 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.460184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.9.215.3:36845)
0811 18:51:20.460435 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.460441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.460453 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:51:20.460495 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.460498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.460514 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.460599 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:20.460829 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.461667 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.463012 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.464451 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.464872 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.464955 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.464984 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.465027 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":149,"threads_started":1}
I20260811 18:51:20.465451 9683 tablet_service.cc:1467] Processing CreateTablet for tablet 058a00d8f9c44ed3811fa43cc53f88dc (DEFAULT_TABLE table=test-table [id=6c220796d04d48a1b54d1fdfe6ee5629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:20.465778 9683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058a00d8f9c44ed3811fa43cc53f88dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:20.465754 9928 tablet_service.cc:1467] Processing CreateTablet for tablet 058a00d8f9c44ed3811fa43cc53f88dc (DEFAULT_TABLE table=test-table [id=6c220796d04d48a1b54d1fdfe6ee5629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:20.466054 9928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058a00d8f9c44ed3811fa43cc53f88dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:20.468626 10092 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Bootstrap starting.
I20260811 18:51:20.468858 10095 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Bootstrap starting.
I20260811 18:51:20.469388 10092 tablet_bootstrap.cc:654] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:20.469451 10095 tablet_bootstrap.cc:654] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:20.469866 10092 log.cc:824] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:20.469929 10095 log.cc:824] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:20.470672 10092 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: No bootstrap required, opened a new log
I20260811 18:51:20.470732 10092 ts_tablet_manager.cc:1397] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:51:20.471808 10095 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: No bootstrap required, opened a new log
I20260811 18:51:20.471868 10095 ts_tablet_manager.cc:1397] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:51:20.472404 10092 raft_consensus.cc:348] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.472555 10092 raft_consensus.cc:374] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:20.472591 10092 raft_consensus.cc:729] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5418ff2b56e54987b23758470949167c, State: Initialized, Role: FOLLOWER
I20260811 18:51:20.472726 10092 consensus_queue.cc:260] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.473026 10092 ts_tablet_manager.cc:1428] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:51:20.473057 9810 heartbeater.cc:503] Master 127.9.215.62:40635 was elected leader, sending a full tablet report...
I20260811 18:51:20.473498 10095 raft_consensus.cc:348] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.473660 10095 raft_consensus.cc:374] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:20.473714 10095 raft_consensus.cc:729] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b440134bb440bda60c5e0585d97f72, State: Initialized, Role: FOLLOWER
I20260811 18:51:20.473832 10095 consensus_queue.cc:260] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.474130 10095 ts_tablet_manager.cc:1428] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:51:20.474193 10064 heartbeater.cc:503] Master 127.9.215.62:40635 was elected leader, sending a full tablet report...
W20260811 18:51:20.552704 9811 tablet.cc:2369] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:51:20.586381 10100 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:20.586516 10100 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.586896 10100 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131)
I20260811 18:51:20.591037 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.587183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:41085 (local address 127.9.215.1:41131)
0811 18:51:20.587236 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.587238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.587250 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:51:20.587283 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.587284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.587295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.587373 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:20.587559 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.588321 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.589996 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.590631 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.590699 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.590721 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.590910 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.590959 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:51:20.591037 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.587169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:55233 (local address 127.9.215.3:36845)
0811 18:51:20.587250 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.587252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.587339 (+ 87us) server_negotiation.cc:407] Connection header received
0811 18:51:20.587371 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.587372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.587384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.587440 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:20.587588 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.588322 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.589995 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.590614 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.590699 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.590713 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.590910 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.590957 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:51:20.591107 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.587082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.9.215.2:41085)
0811 18:51:20.587143 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.587155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.587201 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.587390 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.587390 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.587392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:20.587488 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.587490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.588435 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.588436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.589873 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.589875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.590728 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.590730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.590827 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.590834 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.590897 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.590967 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":69}
I20260811 18:51:20.591115 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.587093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36845 (local address 127.9.215.2:55233)
0811 18:51:20.587275 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.587283 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:20.587333 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:20.587447 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:20.587448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:20.587449 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:20.587528 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.587529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.588435 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.588436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.589862 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:20.589867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.590728 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.590730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:20.590827 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.590834 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:51:20.590897 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:51:20.590967 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":103,"thread_start_us":118,"threads_started":1}
I20260811 18:51:20.591566 9447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a00d8f9c44ed3811fa43cc53f88dc" candidate_uuid: "5418ff2b56e54987b23758470949167c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" is_pre_election: true
I20260811 18:51:20.591771 9447 raft_consensus.cc:2454] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5418ff2b56e54987b23758470949167c in term 0.
I20260811 18:51:20.591982 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a00d8f9c44ed3811fa43cc53f88dc" candidate_uuid: "5418ff2b56e54987b23758470949167c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b440134bb440bda60c5e0585d97f72" is_pre_election: true
I20260811 18:51:20.592015 9603 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5418ff2b56e54987b23758470949167c, d9b5c236ffcc4bcdaf74270c110481f5; no voters:
I20260811 18:51:20.592130 9964 raft_consensus.cc:2454] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5418ff2b56e54987b23758470949167c in term 0.
I20260811 18:51:20.592191 10100 raft_consensus.cc:2781] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:51:20.592281 10100 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:51:20.592305 10100 raft_consensus.cc:3037] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:20.593050 10100 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.593199 10100 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [CANDIDATE]: Term 1 election: Requested vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131)
I20260811 18:51:20.593415 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a00d8f9c44ed3811fa43cc53f88dc" candidate_uuid: "5418ff2b56e54987b23758470949167c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b440134bb440bda60c5e0585d97f72"
I20260811 18:51:20.593429 9447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a00d8f9c44ed3811fa43cc53f88dc" candidate_uuid: "5418ff2b56e54987b23758470949167c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
I20260811 18:51:20.593508 9447 raft_consensus.cc:3037] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:20.593506 9964 raft_consensus.cc:3037] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:20.594461 9964 raft_consensus.cc:2454] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5418ff2b56e54987b23758470949167c in term 1.
I20260811 18:51:20.594702 9599 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5418ff2b56e54987b23758470949167c, f6b440134bb440bda60c5e0585d97f72; no voters:
I20260811 18:51:20.594745 9447 raft_consensus.cc:2454] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5418ff2b56e54987b23758470949167c in term 1.
I20260811 18:51:20.594827 10100 raft_consensus.cc:2781] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:20.595011 10100 raft_consensus.cc:686] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 1 LEADER]: Becoming Leader. State: Replica: 5418ff2b56e54987b23758470949167c, State: Running, Role: LEADER
I20260811 18:51:20.595103 10100 consensus_queue.cc:237] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:20.596189 9157 catalog_manager.cc:5495] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5418ff2b56e54987b23758470949167c (127.9.215.2). New cstate: current_term: 1 leader_uuid: "5418ff2b56e54987b23758470949167c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } health_report { overall_health: UNKNOWN } } }
I20260811 18:51:20.617038 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.614803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.9.215.2:39043)
0811 18:51:20.614873 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:20.614875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:20.614884 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:51:20.614927 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:20.614928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:20.614949 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:20.615013 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:51:20.615110 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.615644 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.616338 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:20.616529 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:20.616584 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:20.616691 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:51:20.616800 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:51:20.616858 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:20.616908 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 18:51:20.616929 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:51:20.619127 9447 raft_consensus.cc:1261] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Refusing update from remote peer 5418ff2b56e54987b23758470949167c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:20.619127 9964 raft_consensus.cc:1261] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Refusing update from remote peer 5418ff2b56e54987b23758470949167c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:20.619534 10100 consensus_queue.cc:1035] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:20.619701 10103 consensus_queue.cc:1035] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:20.622372 10117 mvcc.cc:205] Tried to move back new op lower bound from 7317398653413134336 to 7317398653318619136. Current Snapshot: MvccSnapshot[applied={T|T < 7317398653413134336 or (T in {7317398653413134336})}]
I20260811 18:51:20.622139 10109 mvcc.cc:205] Tried to move back new op lower bound from 7317398653413134336 to 7317398653318619136. Current Snapshot: MvccSnapshot[applied={T|T < 7317398653413134336}]
I20260811 18:51:20.622855 10116 mvcc.cc:205] Tried to move back new op lower bound from 7317398653413134336 to 7317398653318619136. Current Snapshot: MvccSnapshot[applied={T|T < 7317398653413134336}]
I20260811 18:51:20.625322 9157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:20.625423 9157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:20.626116 9157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Sending DeleteTablet for 3 replicas of tablet 058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:20.626427 9402 tablet_service.cc:1514] Processing DeleteTablet for tablet 058a00d8f9c44ed3811fa43cc53f88dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:20 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:36700
I20260811 18:51:20.626446 9928 tablet_service.cc:1514] Processing DeleteTablet for tablet 058a00d8f9c44ed3811fa43cc53f88dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:20 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:42364
I20260811 18:51:20.626454 9683 tablet_service.cc:1514] Processing DeleteTablet for tablet 058a00d8f9c44ed3811fa43cc53f88dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:20 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:48276
I20260811 18:51:20.626669 9562 tablet_replica.cc:330] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: stopping tablet replica
I20260811 18:51:20.626683 10126 tablet_replica.cc:330] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: stopping tablet replica
I20260811 18:51:20.626746 9562 raft_consensus.cc:2229] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:20.626785 10126 raft_consensus.cc:2229] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:20.626812 9562 raft_consensus.cc:2258] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:20.626868 10126 raft_consensus.cc:2258] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:20.627079 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 9236
I20260811 18:51:20.627257 9562 ts_tablet_manager.cc:1905] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:20.627664 10126 ts_tablet_manager.cc:1905] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:20.629418 10126 ts_tablet_manager.cc:1918] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:20.629511 10126 log.cc:1196] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:20.629827 10126 ts_tablet_manager.cc:1939] T 058a00d8f9c44ed3811fa43cc53f88dc P f6b440134bb440bda60c5e0585d97f72: Deleting consensus metadata
I20260811 18:51:20.630368 9128 catalog_manager.cc:4841] TS f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): tablet 058a00d8f9c44ed3811fa43cc53f88dc (table test-table [id=6c220796d04d48a1b54d1fdfe6ee5629]) successfully deleted
I20260811 18:51:20.630311 10128 tablet_replica.cc:330] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: stopping tablet replica
I20260811 18:51:20.630561 10128 raft_consensus.cc:2229] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:51:20.630726 10128 raft_consensus.cc:2258] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:20.631199 10128 ts_tablet_manager.cc:1905] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:20.632759 10128 ts_tablet_manager.cc:1918] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:20.632825 10128 log.cc:1196] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:20.633065 10128 ts_tablet_manager.cc:1939] T 058a00d8f9c44ed3811fa43cc53f88dc P 5418ff2b56e54987b23758470949167c: Deleting consensus metadata
I20260811 18:51:20.633471 9129 catalog_manager.cc:4841] TS 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): tablet 058a00d8f9c44ed3811fa43cc53f88dc (table test-table [id=6c220796d04d48a1b54d1fdfe6ee5629]) successfully deleted
W20260811 18:51:20.652535 9130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:41131 (error 108)
I20260811 18:51:20.652961 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 9557
I20260811 18:51:20.653302 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.653165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.0.0.1:36706)
0811 18:51:20.653219 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.653260 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:41131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:51:20.653453 9130 catalog_manager.cc:4584] TS d9b5c236ffcc4bcdaf74270c110481f5 (127.9.215.1:41131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 058a00d8f9c44ed3811fa43cc53f88dc: Network error: Client connection negotiation failed: client connection to 127.9.215.1:41131: connect: Connection refused (error 111)
I20260811 18:51:20.674548 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 9814
I20260811 18:51:20.706558 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.706344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.0.0.1:36720)
0811 18:51:20.706418 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.706501 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:41131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 18:51:20.707058 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:20.706952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:41131 (local address 127.0.0.1:36736)
0811 18:51:20.706996 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:20.707030 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:41131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 18:51:20.708613 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 9098
I20260811 18:51:40.725942 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40635
--webserver_interface=127.9.215.62
--webserver_port=45551
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:40635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:40.811335 10131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:40.811633 10131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:40.811672 10131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:40.812882 10131 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:51:40.812927 10131 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:51:40.812938 10131 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:51:40.812947 10131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:51:40.812959 10131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:40.812971 10131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:40.812983 10131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:51:40.812992 10131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:51:40.814553 10131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:40635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:40635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:40.814776 10131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:40.814980 10131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:40.820461 10144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:40.820623 10141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:40.820713 10131 server_base.cc:1034] running on GCE node
W20260811 18:51:40.820461 10140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:40.821055 10131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:40.821372 10131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:40.822525 10131 hybrid_clock.cc:648] HybridClock initialized: now 1786474300822502 us; error 33 us; skew 500 ppm
Aug 11 18:51:40 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:40.829886 10131 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:51:40.830495 10131 webserver.cc:460] Webserver started at http://127.9.215.62:45551/ using document root <none> and password file <none>
I20260811 18:51:40.830744 10131 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:40.830806 10131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:40.832469 10131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:40.833209 10159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:40.833421 10131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:40.833523 10131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "135c0193b15846e5a66ecf8a47f43c29"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "bae51f786b0e7ed6474dfafcc589338c"
server_key_iv: "32be1e534e61b4914589acdab47fb9c0"
server_key_version: "encryptionkey@0"
I20260811 18:51:40.833880 10131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:40.843375 10131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:40.845525 10131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:40.845672 10131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:40.849897 10131 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:40635
I20260811 18:51:40.849934 10270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:40635 every 8 connection(s)
I20260811 18:51:40.850368 10131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:51:40.851987 10271 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:51:40.852281 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10131
I20260811 18:51:40.852636 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:41131
--local_ip_for_outbound_sockets=127.9.215.1
--tserver_master_addrs=127.9.215.62:40635
--webserver_port=32921
--webserver_interface=127.9.215.1
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:51:40.852929 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap starting.
I20260811 18:51:40.862716 10271 log.cc:824] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:40.865563 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:51:40.865931 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Bootstrap complete.
I20260811 18:51:40.868428 10271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:40.868698 10271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135c0193b15846e5a66ecf8a47f43c29, State: Initialized, Role: FOLLOWER
I20260811 18:51:40.868870 10271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:40.868949 10271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:51:40.868999 10271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:51:40.869062 10271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:51:40.870225 10271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:40.870364 10271 leader_election.cc:302] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135c0193b15846e5a66ecf8a47f43c29; no voters:
I20260811 18:51:40.870581 10271 leader_election.cc:288] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:51:40.870656 10277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:51:40.870873 10277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [term 3 LEADER]: Becoming Leader. State: Replica: 135c0193b15846e5a66ecf8a47f43c29, State: Running, Role: LEADER
I20260811 18:51:40.870925 10271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:51:40.870981 10277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } }
I20260811 18:51:40.871291 10279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "135c0193b15846e5a66ecf8a47f43c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } } }
I20260811 18:51:40.871367 10279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:40.871547 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 135c0193b15846e5a66ecf8a47f43c29. Latest consensus state: current_term: 3 leader_uuid: "135c0193b15846e5a66ecf8a47f43c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135c0193b15846e5a66ecf8a47f43c29" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 40635 } } }
I20260811 18:51:40.871606 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:40.871711 10285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:51:40.872561 10285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:51:40.872679 10285 catalog_manager.cc:1259] Loaded cluster ID: 74e13e3a5940461f99dd808f429bd8dc
I20260811 18:51:40.872936 10285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:51:40.874090 10285 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:51:40.874305 10285 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Loaded TSK: 0
I20260811 18:51:40.874351 10285 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 135c0193b15846e5a66ecf8a47f43c29: Loaded TSK: 1
F20260811 18:51:40.874588 10285 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6786cb59ad google::LogMessage::Fail() at ??:0
@ 0x7f6786cb888a google::LogMessage::SendToLog() at ??:0
@ 0x7f6786cb54de google::LogMessage::Flush() at ??:0
@ 0x7f6786cb6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6789422104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f678941fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f678ec0f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f678ec0f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f678ec0f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f678ebd5c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f678ebdfad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6788b830d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6788b78954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f67899dc6db start_thread at ??:0
@ 0x7f67857b361f clone at ??:0
W20260811 18:51:40.954198 10275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:40.954427 10275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:40.954463 10275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:40.955904 10275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:40.955964 10275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:40.956004 10275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:51:40.958252 10275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:41131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=32921
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:40.959021 10275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:40.959344 10275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:40.964759 10356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:40.964759 10354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:40.965106 10360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:40.965241 10275 server_base.cc:1034] running on GCE node
I20260811 18:51:40.965444 10275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:40.965725 10275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:40.966894 10275 hybrid_clock.cc:648] HybridClock initialized: now 1786474300966876 us; error 34 us; skew 500 ppm
Aug 11 18:51:40 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:40.969718 10275 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:51:40.970225 10275 webserver.cc:460] Webserver started at http://127.9.215.1:32921/ using document root <none> and password file <none>
I20260811 18:51:40.970448 10275 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:40.970499 10275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:40.972026 10275 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:40.972882 10375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:40.973096 10275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:40.973223 10275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "d9b5c236ffcc4bcdaf74270c110481f5"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "9eb6a3dde24d9b0d1bed09e8e1c731c5"
server_key_iv: "29165c827efa26b096af074601bba7a5"
server_key_version: "encryptionkey@0"
I20260811 18:51:40.973608 10275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:40.986356 10275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:40.988798 10275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:40.988955 10275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:40.989225 10275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:40.989696 10389 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:51:40.991802 10275 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:51:40.991864 10275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 18:51:40.991902 10275 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:51:40.992619 10275 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:51:40.992656 10275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 18:51:40.992777 10389 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap starting.
I20260811 18:51:40.994801 10389 log.cc:824] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:40.998212 10389 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:51:40.998565 10389 tablet_bootstrap.cc:492] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Bootstrap complete.
I20260811 18:51:40.998754 10389 ts_tablet_manager.cc:1397] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:51:40.999678 10275 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:41131
I20260811 18:51:41.000385 10275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:51:41.000855 10603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:41131 every 8 connection(s)
I20260811 18:51:41.001405 10389 raft_consensus.cc:348] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:41.001821 10389 raft_consensus.cc:729] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b5c236ffcc4bcdaf74270c110481f5, State: Initialized, Role: FOLLOWER
I20260811 18:51:41.001971 10389 consensus_queue.cc:260] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:41.002280 10389 ts_tablet_manager.cc:1428] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.004s
I20260811 18:51:41.009424 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10275
I20260811 18:51:41.009711 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:39043
--local_ip_for_outbound_sockets=127.9.215.2
--tserver_master_addrs=127.9.215.62:40635
--webserver_port=33413
--webserver_interface=127.9.215.2
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:41.131434 10609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:51:41.001239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:33727)
0811 18:51:41.001413 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.001461 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:41.001658 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:41.131106 (+129448us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
W20260811 18:51:41.131944 10606 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:40635 (0 consecutive failures): Network error: Failed to ping master at 127.9.215.62:40635: Client connection negotiation failed: client connection to 127.9.215.62:40635: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:51:41.132328 10609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.132193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:45491)
0811 18:51:41.132260 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.132299 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 18:51:41.132663 10609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.132537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:55207)
0811 18:51:41.132593 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.132634 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 18:51:41.133071 10606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:51:41.135983 10614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:41.136204 10614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:41.136233 10614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:41.137465 10614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:41.137506 10614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:41.137532 10614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:51:41.139034 10614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:39043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=33413
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:41.139259 10614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:41.139503 10614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:41.145016 10626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:41.145030 10623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:41.145016 10619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:41.145357 10614 server_base.cc:1034] running on GCE node
I20260811 18:51:41.145583 10614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:41.145866 10614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:41.147027 10614 hybrid_clock.cc:648] HybridClock initialized: now 1786474301147008 us; error 30 us; skew 500 ppm
Aug 11 18:51:41 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:41.149806 10614 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:51:41.150303 10614 webserver.cc:460] Webserver started at http://127.9.215.2:33413/ using document root <none> and password file <none>
I20260811 18:51:41.150520 10614 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:41.150576 10614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:41.152168 10614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:51:41.152908 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:41.153147 10614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:41.153275 10614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "5418ff2b56e54987b23758470949167c"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "3e4714204b06231201f168637b81079f"
server_key_iv: "da0274f03e6f4830e5f68157412963a1"
server_key_version: "encryptionkey@0"
I20260811 18:51:41.153666 10614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:41.172297 10614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:41.174654 10614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:41.174829 10614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:41.175112 10614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:41.175450 10614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:41.175539 10614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:41.175587 10614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:41.175611 10614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:41.182093 10614 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:39043
I20260811 18:51:41.182160 10880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:39043 every 8 connection(s)
I20260811 18:51:41.182574 10614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:51:41.183863 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.183268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:56545)
0811 18:51:41.183501 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.183606 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
W20260811 18:51:41.184350 10881 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:40635 (0 consecutive failures): Network error: Failed to ping master at 127.9.215.62:40635: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
I20260811 18:51:41.184831 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.184674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:41899)
0811 18:51:41.184743 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.184801 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:51:41.185366 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.185247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:38539)
0811 18:51:41.185313 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.185344 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:51:41.185575 10881 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:51:41.191754 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10614
I20260811 18:51:41.192013 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:36845
--local_ip_for_outbound_sockets=127.9.215.3
--tserver_master_addrs=127.9.215.62:40635
--webserver_port=45541
--webserver_interface=127.9.215.3
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:45975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:41.279656 10885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:41.279886 10885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:41.279923 10885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:41.281116 10885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:41.281163 10885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:41.281203 10885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:51:41.282748 10885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:45975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:36845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=45541
--tserver_master_addrs=127.9.215.62:40635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:41.283002 10885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:41.283241 10885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:41.288575 10901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:41.288621 10895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:41.288594 10894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:41.288627 10885 server_base.cc:1034] running on GCE node
I20260811 18:51:41.289006 10885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:41.289258 10885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:41.290398 10885 hybrid_clock.cc:648] HybridClock initialized: now 1786474301290384 us; error 33 us; skew 500 ppm
Aug 11 18:51:41 dist-test-slave-pww0 krb5kdc[8017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:41.292935 10885 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:51:41.293434 10885 webserver.cc:460] Webserver started at http://127.9.215.3:45541/ using document root <none> and password file <none>
I20260811 18:51:41.293648 10885 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:41.293699 10885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:41.295043 10885 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:41.295691 10913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:41.295853 10885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:51:41.295943 10885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "f6b440134bb440bda60c5e0585d97f72"
format_stamp: "Formatted at 2026-08-11 18:50:59 on dist-test-slave-pww0"
server_key: "a5c2a04c83a55758e416cdcedf803ebb"
server_key_iv: "137c977d78d5a636d858a9b02e4f96ac"
server_key_version: "encryptionkey@0"
I20260811 18:51:41.296308 10885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:41.306196 10885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:41.308519 10885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:41.308688 10885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:41.308928 10885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:41.309253 10885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:41.309289 10885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:41.309319 10885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:41.309343 10885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:41.315562 10885 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:36845
I20260811 18:51:41.315618 11128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:36845 every 8 connection(s)
I20260811 18:51:41.315994 10885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:51:41.317106 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.316702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:43753)
0811 18:51:41.316860 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.316948 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
W20260811 18:51:41.317579 11135 heartbeater.cc:644] Failed to heartbeat to 127.9.215.62:40635 (0 consecutive failures): Network error: Failed to ping master at 127.9.215.62:40635: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
I20260811 18:51:41.317910 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.317818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:51431)
0811 18:51:41.317861 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.317890 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 18:51:41.318270 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 10885
I20260811 18:51:41.318528 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:41.318148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:52931)
0811 18:51:41.318476 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:41.318504 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309}
W20260811 18:51:41.318708 11135 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:51:42.133847 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.133548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:58197)
0811 18:51:42.133703 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:42.133776 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260811 18:51:42.186305 11141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.186006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:40641)
0811 18:51:42.186182 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:42.186234 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260811 18:51:42.319299 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.319044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:49993)
0811 18:51:42.319179 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:42.319219 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260811 18:51:42.818961 11144 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:42.819109 11144 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:42.819501 11144 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
Aug 11 18:51:42 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.3@KRBTEST.COM
Aug 11 18:51:42 dist-test-slave-pww0 krb5kdc[8017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.2@KRBTEST.COM
I20260811 18:51:42.830232 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.819834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:58061 (local address 127.9.215.3:36845)
0811 18:51:42.819989 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:42.819999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:42.820022 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:51:42.820458 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:42.820460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:42.820504 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:42.820646 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:42.821478 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.822483 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.823604 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.824020 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.824123 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:42.826424 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:42.826430 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:42.826430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:42.826445 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:42.828415 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:42.828960 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:42.828964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:42.828964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:42.829056 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:42.829310 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:42.829311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:42.829311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:42.829592 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:42.829670 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:42.829978 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 18:51:42.830110 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":45,"threads_started":1}
I20260811 18:51:42.830226 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.819833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:58373 (local address 127.9.215.2:39043)
0811 18:51:42.819981 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:42.819994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:42.820020 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:51:42.820474 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:42.820476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:42.820516 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:42.820645 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:42.821677 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.822563 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.823603 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.824021 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.824123 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:42.826901 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:42.826904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:42.826904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:42.826916 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:42.828491 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:42.828953 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:42.828956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:42.828957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:42.829053 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:42.829303 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:42.829303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:42.829304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:42.829592 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:42.829670 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:42.829975 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 18:51:42.830103 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":43,"threads_started":1}
I20260811 18:51:42.830226 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.819703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36845 (local address 127.9.215.1:58061)
0811 18:51:42.819884 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:42.819896 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:42.819962 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:42.820706 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:42.820709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:42.821098 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:42.821330 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:42.821337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.822593 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.822594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:42.823518 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:42.823521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.824109 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.824110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:42.824238 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:42.824832 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:42.824836 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:42.826298 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:42.828498 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:42.828510 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:42.828513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:42.828877 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:42.829120 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:42.829121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:42.829121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:42.829248 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:42.829663 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:42.829672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:42.829809 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 18:51:42.829902 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:51:42.830114 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":45,"thread_start_us":107,"threads_started":1}
I20260811 18:51:42.830334 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:42.819703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:39043 (local address 127.9.215.1:58373)
0811 18:51:42.819897 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:42.819908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:42.819968 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:42.820707 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:42.820709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:42.821099 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:42.821528 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:42.821531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.822669 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.822670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:42.823522 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:42.823525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:42.824111 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:42.824112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:42.824237 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:42.824790 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:42.824798 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:42.826788 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:42.828578 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:42.828579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:42.828580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:42.828875 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:42.829117 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:42.829118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:42.829118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:42.829242 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:42.829664 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:42.829672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:42.829811 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 18:51:42.829902 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 18:51:42.830192 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":152,"spinlock_wait_cycles":12032,"thread_start_us":126,"threads_started":1}
W20260811 18:51:42.831669 10376 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
W20260811 18:51:42.831758 10378 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:42.831830 10378 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5; no voters: 5418ff2b56e54987b23758470949167c, f6b440134bb440bda60c5e0585d97f72
I20260811 18:51:42.831948 11144 raft_consensus.cc:2726] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:51:43.134568 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:43.134405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:36625)
0811 18:51:43.134468 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:43.134511 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 18:51:43.187157 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:43.186882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:57787)
0811 18:51:43.187032 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:43.187099 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260811 18:51:43.320089 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:43.319808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:39139)
0811 18:51:43.319976 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:43.320035 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260811 18:51:44.135355 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:44.135116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:56973)
0811 18:51:44.135257 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:44.135300 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260811 18:51:44.188038 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:44.187807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:41371)
0811 18:51:44.187938 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:44.187986 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260811 18:51:44.320891 11171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:44.320642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:42865)
0811 18:51:44.320791 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:44.320840 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260811 18:51:45.072182 11172 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:45.072332 11172 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:45.072525 11172 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
W20260811 18:51:45.073125 10376 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
W20260811 18:51:45.073225 10378 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:45.073269 10378 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5; no voters: 5418ff2b56e54987b23758470949167c, f6b440134bb440bda60c5e0585d97f72
I20260811 18:51:45.073346 11172 raft_consensus.cc:2726] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:51:45.136202 11174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:45.135894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:40185)
0811 18:51:45.136086 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:45.136136 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
I20260811 18:51:45.188819 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:45.188550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:50049)
0811 18:51:45.188695 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:45.188752 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260811 18:51:45.321689 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:45.321388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:57053)
0811 18:51:45.321556 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:45.321602 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260811 18:51:46.137033 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:46.136764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:51357)
0811 18:51:46.136944 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:46.136984 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260811 18:51:46.189755 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:46.189431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:49405)
0811 18:51:46.189628 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:46.189683 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260811 18:51:46.322495 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:46.322205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:45993)
0811 18:51:46.322390 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:46.322440 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260811 18:51:46.610688 11187 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:46.610809 11187 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:46.611011 11187 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
W20260811 18:51:46.611552 10378 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
W20260811 18:51:46.611653 10376 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:46.611698 10376 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5; no voters: 5418ff2b56e54987b23758470949167c, f6b440134bb440bda60c5e0585d97f72
I20260811 18:51:46.611773 11187 raft_consensus.cc:2726] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:51:47.137725 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:47.137456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:45245)
0811 18:51:47.137625 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:47.137674 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260811 18:51:47.190611 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:47.190331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:50839)
0811 18:51:47.190511 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:47.190559 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260811 18:51:47.323340 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:47.323084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:43633)
0811 18:51:47.323241 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:47.323289 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260811 18:51:48.138513 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:48.138255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:48555)
0811 18:51:48.138414 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:48.138462 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260811 18:51:48.191398 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:48.191143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:41839)
0811 18:51:48.191297 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:48.191346 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260811 18:51:48.324116 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:48.323882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:59905)
0811 18:51:48.324028 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:48.324064 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260811 18:51:48.380645 11199 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:48.380775 11199 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:48.380950 11199 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
W20260811 18:51:48.381407 10378 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
W20260811 18:51:48.381505 10376 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:48.381587 10376 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5; no voters: 5418ff2b56e54987b23758470949167c, f6b440134bb440bda60c5e0585d97f72
I20260811 18:51:48.381662 11199 raft_consensus.cc:2726] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:51:49.139308 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:49.139067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:49659)
0811 18:51:49.139213 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:49.139257 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260811 18:51:49.192318 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:49.192040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:51671)
0811 18:51:49.192214 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:49.192265 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260811 18:51:49.324888 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:49.324631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:60509)
0811 18:51:49.324783 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:49.324837 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260811 18:51:50.140089 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:50.139811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:56175)
0811 18:51:50.139990 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:50.140037 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
I20260811 18:51:50.193198 11219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:50.192923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:48707)
0811 18:51:50.193094 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:50.193141 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260811 18:51:50.325734 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:50.325458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:49411)
0811 18:51:50.325621 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:50.325676 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
I20260811 18:51:51.140954 11222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:51.140696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:41959)
0811 18:51:51.140860 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:51.140901 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260811 18:51:51.194242 11225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:51.193962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:51227)
0811 18:51:51.194151 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:51.194192 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260811 18:51:51.326574 11226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:51.326312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:38723)
0811 18:51:51.326472 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:51.326520 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260811 18:51:52.141929 11232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:52.141669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:50227)
0811 18:51:52.141830 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:52.141880 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260811 18:51:52.195223 11233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:52.194917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:36333)
0811 18:51:52.195111 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:52.195163 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
I20260811 18:51:52.327360 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:52.327127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:59237)
0811 18:51:52.327265 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:52.327305 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260811 18:51:53.142833 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:53.142544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:48051)
0811 18:51:53.142731 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:53.142780 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260811 18:51:53.196054 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:53.195795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:46153)
0811 18:51:53.195951 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:53.196001 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260811 18:51:53.328276 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:53.327974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:52693)
0811 18:51:53.328168 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:53.328220 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260811 18:51:53.550945 11249 raft_consensus.cc:482] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:53.551054 11249 raft_consensus.cc:504] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b440134bb440bda60c5e0585d97f72" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36845 } } peers { permanent_uuid: "5418ff2b56e54987b23758470949167c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 39043 } } peers { permanent_uuid: "d9b5c236ffcc4bcdaf74270c110481f5" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 41131 } }
I20260811 18:51:53.551225 11249 leader_election.cc:288] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845), 5418ff2b56e54987b23758470949167c (127.9.215.2:39043)
W20260811 18:51:53.551752 10378 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5418ff2b56e54987b23758470949167c (127.9.215.2:39043): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
W20260811 18:51:53.551859 10376 leader_election.cc:341] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6b440134bb440bda60c5e0585d97f72 (127.9.215.3:36845): Not found: Tablet not found: 058a00d8f9c44ed3811fa43cc53f88dc
I20260811 18:51:53.551905 10376 leader_election.cc:302] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9b5c236ffcc4bcdaf74270c110481f5; no voters: 5418ff2b56e54987b23758470949167c, f6b440134bb440bda60c5e0585d97f72
I20260811 18:51:53.551980 11249 raft_consensus.cc:2726] T 058a00d8f9c44ed3811fa43cc53f88dc P d9b5c236ffcc4bcdaf74270c110481f5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:51:54.143745 11252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:54.143484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:36709)
0811 18:51:54.143641 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:54.143691 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260811 18:51:54.196907 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:54.196621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:57129)
0811 18:51:54.196804 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:54.196855 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260811 18:51:54.329146 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:54.328859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:46853)
0811 18:51:54.329041 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:54.329094 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":91,"threads_started":1}
I20260811 18:51:55.144560 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:55.144309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:38029)
0811 18:51:55.144455 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:55.144507 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260811 18:51:55.197880 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:55.197552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:43735)
0811 18:51:55.197771 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:55.197825 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260811 18:51:55.330088 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:55.329791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:39153)
0811 18:51:55.329980 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:55.330030 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260811 18:51:56.147854 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:56.145305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.1:48121)
0811 18:51:56.147737 (+ 2432us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:56.147796 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2409,"thread_start_us":2366,"threads_started":1}
I20260811 18:51:56.198849 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:56.198565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.2:60359)
0811 18:51:56.198749 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:56.198797 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260811 18:51:56.322453 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10275
I20260811 18:51:56.335014 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:56.332278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:40635 (local address 127.9.215.3:34667)
0811 18:51:56.334874 (+ 2596us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:56.334943 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.62:40635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2568,"thread_start_us":2466,"threads_started":1}
I20260811 18:51:56.340720 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10614
I20260811 18:51:56.376310 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 10885
2026-08-11T18:51:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58768 ms)
[----------] 30 tests from SecurityITest (248280 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:51:56 dist-test-slave-pww0 krb5kdc[11275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:51:56 dist-test-slave-pww0 krb5kdc[11275](info): set up 2 sockets
Aug 11 18:51:56 dist-test-slave-pww0 krb5kdc[11275](info): commencing operation
krb5kdc: starting...
W20260811 18:51:57.623504 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:51:57 dist-test-slave-pww0 krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:51:57.763923 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:39643
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:39643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:57.850337 11311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:57.850591 11311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:57.850629 11311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:57.851990 11311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:51:57.852058 11311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:57.852078 11311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:57.852095 11311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:51:57.852113 11311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:51:57.854199 11311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:39643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:39643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:57.854549 11311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:57.854797 11311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:51:57.860780 11311 server_base.cc:1034] running on GCE node
W20260811 18:51:57.860747 11320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:57.860747 11325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:57.860957 11323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:57.861420 11311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:57.861773 11311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:57.862964 11311 hybrid_clock.cc:648] HybridClock initialized: now 1786474317862938 us; error 36 us; skew 500 ppm
Aug 11 18:51:57 dist-test-slave-pww0 krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474317, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:57.865764 11311 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:51:57.866298 11311 webserver.cc:460] Webserver started at http://127.9.215.62:42021/ using document root <none> and password file <none>
I20260811 18:51:57.866504 11311 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:57.866621 11311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:57.866801 11311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:57.868002 11311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "aea322dd507a4e839b07c7cfb0d47545"
format_stamp: "Formatted at 2026-08-11 18:51:57 on dist-test-slave-pww0"
server_key: "29648b39d9aa6fb62a98fb615756e1a6"
server_key_iv: "7c4901030480d20bdb314630992025f6"
server_key_version: "encryptionkey@0"
I20260811 18:51:57.868330 11311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "aea322dd507a4e839b07c7cfb0d47545"
format_stamp: "Formatted at 2026-08-11 18:51:57 on dist-test-slave-pww0"
server_key: "29648b39d9aa6fb62a98fb615756e1a6"
server_key_iv: "7c4901030480d20bdb314630992025f6"
server_key_version: "encryptionkey@0"
I20260811 18:51:57.869870 11311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:51:57.870818 11339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:57.871055 11311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:57.871186 11311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "aea322dd507a4e839b07c7cfb0d47545"
format_stamp: "Formatted at 2026-08-11 18:51:57 on dist-test-slave-pww0"
server_key: "29648b39d9aa6fb62a98fb615756e1a6"
server_key_iv: "7c4901030480d20bdb314630992025f6"
server_key_version: "encryptionkey@0"
I20260811 18:51:57.871269 11311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:57.901741 11311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:57.904259 11311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:57.904425 11311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:57.908910 11311 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:39643
I20260811 18:51:57.908993 11449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:39643 every 8 connection(s)
I20260811 18:51:57.909358 11311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:51:57.910295 11450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:57.910853 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11311
I20260811 18:51:57.910949 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:51:57.911128 10076 external_mini_cluster.cc:1428] Setting key 034ea113f380459c00b2d14b7d7ccb8c
I20260811 18:51:57.913371 11450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545: Bootstrap starting.
I20260811 18:51:57.914114 11450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:57.914455 11450 log.cc:824] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:57.915239 11450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545: No bootstrap required, opened a new log
I20260811 18:51:57.917201 11450 raft_consensus.cc:348] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea322dd507a4e839b07c7cfb0d47545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39643 } }
I20260811 18:51:57.917352 11450 raft_consensus.cc:374] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:57.917375 11450 raft_consensus.cc:729] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aea322dd507a4e839b07c7cfb0d47545, State: Initialized, Role: FOLLOWER
I20260811 18:51:57.917490 11450 consensus_queue.cc:260] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea322dd507a4e839b07c7cfb0d47545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39643 } }
I20260811 18:51:57.917562 11450 raft_consensus.cc:388] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:51:57.917593 11450 raft_consensus.cc:482] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:51:57.917644 11450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 18:51:57 dist-test-slave-pww0 krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:57.918289 11450 raft_consensus.cc:504] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea322dd507a4e839b07c7cfb0d47545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39643 } }
I20260811 18:51:57.918411 11450 leader_election.cc:302] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aea322dd507a4e839b07c7cfb0d47545; no voters:
I20260811 18:51:57.918627 11450 leader_election.cc:288] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:51:57.918689 11461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:57.918857 11461 raft_consensus.cc:686] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [term 1 LEADER]: Becoming Leader. State: Replica: aea322dd507a4e839b07c7cfb0d47545, State: Running, Role: LEADER
I20260811 18:51:57.918917 11450 sys_catalog.cc:564] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:51:57.919009 11461 consensus_queue.cc:237] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea322dd507a4e839b07c7cfb0d47545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39643 } }
I20260811 18:51:57.919420 11461 sys_catalog.cc:455] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aea322dd507a4e839b07c7cfb0d47545. Latest consensus state: current_term: 1 leader_uuid: "aea322dd507a4e839b07c7cfb0d47545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea322dd507a4e839b07c7cfb0d47545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39643 } } }
I20260811 18:51:57.919494 11462 sys_catalog.cc:455] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aea322dd507a4e839b07c7cfb0d47545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aea322dd507a4e839b07c7cfb0d47545" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 39643 } } }
I20260811 18:51:57.919682 11462 sys_catalog.cc:458] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:57.919884 11461 sys_catalog.cc:458] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545 [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:57.920058 11479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:51:57.920892 11479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:51:57.922456 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:57.911680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.9.215.62:39643)
0811 18:51:57.911897 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:57.911907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:57.911944 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:51:57.912473 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:57.912483 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:57.912550 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:57.912714 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:57.913326 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:57.914671 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:57.915536 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:57.915995 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:57.916097 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:57.918574 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:57.918580 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:57.918580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:57.918595 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:57.920736 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:57.921174 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:57.921177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:57.921178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:57.921287 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:57.921524 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:57.921525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:57.921525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:57.921815 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:57.921969 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:57.922099 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:51:57.922239 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260811 18:51:57.923179 11479 catalog_manager.cc:1347] Generated new cluster ID: 8439aea992f748398d94922595746c58
I20260811 18:51:57.923234 11479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:51:57.941601 11479 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:51:57.942406 11479 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:51:57.949944 11479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545: Generated new TSK 0
I20260811 18:51:57.950222 11479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:51:57.991387 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:39643
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:58.080379 11508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:58.080626 11508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:58.080662 11508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:58.081950 11508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:58.082015 11508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:58.082054 11508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:51:58.083763 11508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:39643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:58.084066 11508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:58.084328 11508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:58.089915 11519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:58.090161 11518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:58.090232 11508 server_base.cc:1034] running on GCE node
W20260811 18:51:58.090169 11524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:58.090593 11508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:58.090912 11508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:58.092139 11508 hybrid_clock.cc:648] HybridClock initialized: now 1786474318092121 us; error 31 us; skew 500 ppm
Aug 11 18:51:58 dist-test-slave-pww0 krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474318, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:58.095178 11508 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:51:58.095769 11508 webserver.cc:460] Webserver started at http://127.9.215.1:41747/ using document root <none> and password file <none>
I20260811 18:51:58.095999 11508 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:58.096052 11508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:58.096194 11508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:58.097308 11508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "2f24ba3ddb524c6c86407366abeadfa3"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "e134c744a005b2c8baabae1831a8bfaf"
server_key_iv: "f74cac4d45a378683bcb1397bf885e8d"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.097654 11508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "2f24ba3ddb524c6c86407366abeadfa3"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "e134c744a005b2c8baabae1831a8bfaf"
server_key_iv: "f74cac4d45a378683bcb1397bf885e8d"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.099015 11508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:51:58.099910 11540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.100199 11508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:58.100301 11508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "2f24ba3ddb524c6c86407366abeadfa3"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "e134c744a005b2c8baabae1831a8bfaf"
server_key_iv: "f74cac4d45a378683bcb1397bf885e8d"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.100369 11508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:58.118242 11508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:58.120848 11508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:58.121011 11508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:58.121292 11508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:58.121637 11508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:58.121685 11508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.121716 11508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:58.121732 11508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.127948 11508 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:42267
I20260811 18:51:58.128093 11797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:42267 every 8 connection(s)
I20260811 18:51:58.128391 11508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:51:58 dist-test-slave-pww0 krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474318, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:58.138434 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11508
I20260811 18:51:58.138533 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:51:58.138726 10076 external_mini_cluster.cc:1428] Setting key cb1eed6e8a2f98e2908184321b829585
I20260811 18:51:58.139233 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.129237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:44511 (local address 127.9.215.62:39643)
0811 18:51:58.129349 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.129351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.129542 (+ 191us) server_negotiation.cc:407] Connection header received
0811 18:51:58.130132 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.130133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.130148 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.130214 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:58.131385 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.131978 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.133156 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.133383 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.133456 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.136038 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:58.136040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:58.136040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:58.136049 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:58.137513 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.137999 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.138001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.138001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.138059 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.138388 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.138389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.138390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.138553 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:58.138649 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.138995 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.139137 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:51:58.139438 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.129133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:39643 (local address 127.9.215.1:44511)
0811 18:51:58.129378 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.129434 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.129988 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.130341 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.130345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.130998 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:58.131238 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.131244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.132107 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.132109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.133020 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.133023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.133496 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.133497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.133656 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.134296 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:58.134306 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:58.135906 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:58.137628 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:58.137640 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:58.137643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:58.137910 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:58.138130 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:58.138131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:58.138131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:58.138288 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:58.138661 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:58.138672 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:58.138838 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.138959 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.139176 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":99,"threads_started":1}
I20260811 18:51:58.140100 11799 heartbeater.cc:347] Connected to a master server at 127.9.215.62:39643
I20260811 18:51:58.140221 11799 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:58.140432 11799 heartbeater.cc:511] Master 127.9.215.62:39643 requested a full tablet report, sending...
I20260811 18:51:58.141110 11370 ts_manager.cc:194] Registered new tserver with Master: 2f24ba3ddb524c6c86407366abeadfa3 (127.9.215.1:42267)
I20260811 18:51:58.141924 11370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:44511
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:51:58.172834 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:39643
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:58.263928 11813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:58.264183 11813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:58.264221 11813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:58.265539 11813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:58.265600 11813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:58.265640 11813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:51:58.267359 11813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:39643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:58.267689 11813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:58.267951 11813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:58.273622 11824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:58.273825 11813 server_base.cc:1034] running on GCE node
W20260811 18:51:58.273702 11820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:58.273626 11821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:58.274142 11813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:58.274422 11813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:58.275549 11813 hybrid_clock.cc:648] HybridClock initialized: now 1786474318275530 us; error 31 us; skew 500 ppm
Aug 11 18:51:58 dist-test-slave-pww0 krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474318, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:58.278667 11813 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:51:58.279371 11813 webserver.cc:460] Webserver started at http://127.9.215.2:33183/ using document root <none> and password file <none>
I20260811 18:51:58.279767 11813 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:58.279836 11813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:58.279973 11813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:58.281108 11813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "929b3c449956469d9680d0cfb34df6f5"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "cd3f129dc454065983ab46ae702c47ae"
server_key_iv: "84f63c97295f87b6ccee32df376bca12"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.281477 11813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "929b3c449956469d9680d0cfb34df6f5"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "cd3f129dc454065983ab46ae702c47ae"
server_key_iv: "84f63c97295f87b6ccee32df376bca12"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.282979 11813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:51:58.284106 11833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.284463 11813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:51:58.284577 11813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "929b3c449956469d9680d0cfb34df6f5"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "cd3f129dc454065983ab46ae702c47ae"
server_key_iv: "84f63c97295f87b6ccee32df376bca12"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.284667 11813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:58.302376 11813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:58.304872 11813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:58.305043 11813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:58.305311 11813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:58.305652 11813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:58.305689 11813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.305721 11813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:58.305751 11813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.311617 11813 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:40055
I20260811 18:51:58.311672 12054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:40055 every 8 connection(s)
I20260811 18:51:58.312014 11813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:51:58 dist-test-slave-pww0 krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474318, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:58.319936 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 11813
I20260811 18:51:58.320230 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:51:58.320773 10076 external_mini_cluster.cc:1428] Setting key e71538b7ee7e2c73a9816c845a066d84
I20260811 18:51:58.323944 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.312886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:59811 (local address 127.9.215.62:39643)
0811 18:51:58.312956 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.312958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.313193 (+ 235us) server_negotiation.cc:407] Connection header received
0811 18:51:58.313736 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.313738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.313752 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.313816 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:58.315060 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.315626 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.316675 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.316874 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.316937 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.319848 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:58.319851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:58.319851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:58.319861 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:58.322185 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.322698 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.322700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.322700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.322783 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.323109 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.323109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.323110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.323260 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:58.323330 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.323787 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.323874 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:51:58.324236 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.312828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:39643 (local address 127.9.215.2:59811)
0811 18:51:58.313034 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.313106 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.313627 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.313898 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.313903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.314660 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:58.314924 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.314930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.315720 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.315721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.316573 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.316576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.316959 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.316960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.317103 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.317783 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:58.317793 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:58.319712 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:58.322317 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:58.322329 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:58.322332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:58.322620 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:58.322877 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:58.322878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:58.322879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:58.323043 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:58.323358 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:58.323370 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:58.323624 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.323765 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.323972 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260811 18:51:58.324820 12055 heartbeater.cc:347] Connected to a master server at 127.9.215.62:39643
I20260811 18:51:58.324927 12055 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:58.325126 12055 heartbeater.cc:511] Master 127.9.215.62:39643 requested a full tablet report, sending...
I20260811 18:51:58.325552 11370 ts_manager.cc:194] Registered new tserver with Master: 929b3c449956469d9680d0cfb34df6f5 (127.9.215.2:40055)
I20260811 18:51:58.326048 11370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:59811
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:51:58.351646 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:39643
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:51:58.442744 12064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:58.442994 12064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:58.443020 12064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:58.444267 12064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:58.444321 12064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:58.444348 12064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:51:58.446250 12064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:39643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:58.446503 12064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:58.446738 12064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:58.455180 12074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:58.455189 12076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:58.455515 12064 server_base.cc:1034] running on GCE node
W20260811 18:51:58.455189 12078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:58.455885 12064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:58.456161 12064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:58.457345 12064 hybrid_clock.cc:648] HybridClock initialized: now 1786474318457307 us; error 46 us; skew 500 ppm
Aug 11 18:51:58 dist-test-slave-pww0 krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474318, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:51:58.460175 12064 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:51:58.460649 12064 webserver.cc:460] Webserver started at http://127.9.215.3:37001/ using document root <none> and password file <none>
I20260811 18:51:58.460886 12064 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:58.460934 12064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:58.461066 12064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:58.462196 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "dbe7d0bab60049d4bc1c62bf598dc108"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "c06c00e7f9e427196532dfa868a24c6f"
server_key_iv: "066d65dadc7f96ae82e4f6fec72b6160"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.462543 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "dbe7d0bab60049d4bc1c62bf598dc108"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "c06c00e7f9e427196532dfa868a24c6f"
server_key_iv: "066d65dadc7f96ae82e4f6fec72b6160"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.464116 12064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:51:58.465070 12092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.465354 12064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:51:58.465466 12064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "dbe7d0bab60049d4bc1c62bf598dc108"
format_stamp: "Formatted at 2026-08-11 18:51:58 on dist-test-slave-pww0"
server_key: "c06c00e7f9e427196532dfa868a24c6f"
server_key_iv: "066d65dadc7f96ae82e4f6fec72b6160"
server_key_version: "encryptionkey@0"
I20260811 18:51:58.465541 12064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:58.487210 12064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:58.489645 12064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:58.489782 12064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:58.490022 12064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:58.490376 12064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:58.490414 12064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.490446 12064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:58.490464 12064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:58.496724 12064 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:45867
I20260811 18:51:58.496814 12340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:45867 every 8 connection(s)
I20260811 18:51:58.497123 12064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:51:58.498580 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12064
I20260811 18:51:58.498670 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:51:58.498849 10076 external_mini_cluster.cc:1428] Setting key ea462acdd3ce0d334f18f58242886645
Aug 11 18:51:58 dist-test-slave-pww0 krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474318, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:51:58.506969 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.498187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:60169 (local address 127.9.215.62:39643)
0811 18:51:58.498242 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.498244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.498786 (+ 542us) server_negotiation.cc:407] Connection header received
0811 18:51:58.498998 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.499000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.499013 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.499065 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:58.500113 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.500631 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.501649 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.501852 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.501908 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.504236 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:58.504237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:58.504238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:58.504247 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:58.505543 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.505997 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.505999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.505999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.506057 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.506297 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.506298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.506298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.506422 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:58.506483 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.506776 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.506870 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:51:58.507171 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.498127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:39643 (local address 127.9.215.3:60169)
0811 18:51:58.498315 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.498361 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.498884 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.499144 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.499149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.499800 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:58.500015 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.500021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.500730 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.500731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.501562 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.501564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.501927 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.501928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.502047 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.502584 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:51:58.502594 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:58.504127 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:58.505632 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:58.505642 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:58.505645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:58.505920 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:58.506113 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:51:58.506114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:51:58.506114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:51:58.506237 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:51:58.506480 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:58.506488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:58.506643 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.506746 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.506929 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260811 18:51:58.507795 12341 heartbeater.cc:347] Connected to a master server at 127.9.215.62:39643
I20260811 18:51:58.507905 12341 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:58.508081 12341 heartbeater.cc:511] Master 127.9.215.62:39643 requested a full tablet report, sending...
I20260811 18:51:58.508467 11370 ts_manager.cc:194] Registered new tserver with Master: dbe7d0bab60049d4bc1c62bf598dc108 (127.9.215.3:45867)
I20260811 18:51:58.508898 11370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:60169
I20260811 18:51:58.510447 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:51:58.518010 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.511818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.9.215.62:39643)
0811 18:51:58.511876 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.511878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.511965 (+ 87us) server_negotiation.cc:407] Connection header received
0811 18:51:58.512028 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.512029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.512038 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.512077 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:58.512614 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.513121 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.513882 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.514087 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.514168 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.515517 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:58.515519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:58.515519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:51:58.515529 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:58.516875 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.517237 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.517238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.517238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.517299 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:51:58.517442 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:51:58.517443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:51:58.517443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:51:58.517601 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:58.517698 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.517866 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.517939 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:51:58.519796 11370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:51:58.521495 11370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:51:58.528729 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.524403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42267 (local address 127.0.0.1:52230)
0811 18:51:58.524762 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.524811 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.524889 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.525115 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.525119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.525121 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:58.525205 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.525208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.526359 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.526361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.527696 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.527698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.528393 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.528395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.528489 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.528505 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.528576 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.528632 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260811 18:51:58.528743 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.524518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.9.215.1:42267)
0811 18:51:58.524940 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.524945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.524955 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:51:58.525000 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.525003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.525019 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.525104 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:58.525355 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.526249 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.527790 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.528310 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.528380 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.528451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.528596 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.528640 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":99,"threads_started":1}
I20260811 18:51:58.529425 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.524405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:45867 (local address 127.0.0.1:58028)
0811 18:51:58.524773 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.524813 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.524889 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.525279 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.525279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.525281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:58.525317 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.525318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.527508 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.527509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.528309 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.528311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.529154 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.529155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.529239 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.529244 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.529289 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.529333 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":74,"threads_started":1}
I20260811 18:51:58.529435 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.524503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.9.215.3:45867)
0811 18:51:58.524827 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.524835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.524965 (+ 130us) server_negotiation.cc:407] Connection header received
0811 18:51:58.525003 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.525005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.525019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.525099 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:58.525398 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.526660 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.528406 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.529072 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.529141 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.529213 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.529315 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.529356 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":140,"threads_started":1}
I20260811 18:51:58.530011 11649 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb63954523146e58f22f4e404a96cd2 (DEFAULT_TABLE table=test-table [id=e1b074b5114646f18277e60752bf8d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:58.530485 11649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb63954523146e58f22f4e404a96cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:58.530475 12204 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb63954523146e58f22f4e404a96cd2 (DEFAULT_TABLE table=test-table [id=e1b074b5114646f18277e60752bf8d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:58.530844 12204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb63954523146e58f22f4e404a96cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:58.532060 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.524629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:40055 (local address 127.0.0.1:50192)
0811 18:51:58.524960 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.524969 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.525009 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.525340 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.525341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.525342 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:58.525385 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.525386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.526783 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.526784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.528741 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.528743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.531740 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.531741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.531844 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.531851 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.531902 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.531971 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":7,"thread_start_us":79,"threads_started":1}
I20260811 18:51:58.532091 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.524783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.9.215.2:40055)
0811 18:51:58.525115 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.525120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.525129 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:51:58.525166 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.525168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.525183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.525259 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:58.525456 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.526652 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.528834 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.531613 (+ 2779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.531710 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.531795 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.531924 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.531981 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":86,"threads_started":1}
I20260811 18:51:58.532999 11934 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb63954523146e58f22f4e404a96cd2 (DEFAULT_TABLE table=test-table [id=e1b074b5114646f18277e60752bf8d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:58.533358 11934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb63954523146e58f22f4e404a96cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:58.534493 12376 tablet_bootstrap.cc:492] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Bootstrap starting.
I20260811 18:51:58.534755 12377 tablet_bootstrap.cc:492] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: Bootstrap starting.
I20260811 18:51:58.535550 12377 tablet_bootstrap.cc:654] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:58.536180 12376 tablet_bootstrap.cc:654] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:58.536187 12377 log.cc:824] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:58.536772 12376 log.cc:824] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:58.536799 12380 tablet_bootstrap.cc:492] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Bootstrap starting.
I20260811 18:51:58.537420 12380 tablet_bootstrap.cc:654] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:58.537606 12376 tablet_bootstrap.cc:492] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: No bootstrap required, opened a new log
I20260811 18:51:58.537671 12376 ts_tablet_manager.cc:1397] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:51:58.537907 12380 log.cc:824] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:58.538575 12380 tablet_bootstrap.cc:492] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: No bootstrap required, opened a new log
I20260811 18:51:58.538645 12380 ts_tablet_manager.cc:1397] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:51:58.538754 12377 tablet_bootstrap.cc:492] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: No bootstrap required, opened a new log
I20260811 18:51:58.538841 12377 ts_tablet_manager.cc:1397] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:51:58.539870 12376 raft_consensus.cc:348] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.540086 12376 raft_consensus.cc:374] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:58.540126 12376 raft_consensus.cc:729] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe7d0bab60049d4bc1c62bf598dc108, State: Initialized, Role: FOLLOWER
I20260811 18:51:58.540314 12376 consensus_queue.cc:260] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.540266 12380 raft_consensus.cc:348] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.540434 12380 raft_consensus.cc:374] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:58.540493 12380 raft_consensus.cc:729] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929b3c449956469d9680d0cfb34df6f5, State: Initialized, Role: FOLLOWER
I20260811 18:51:58.540655 12376 ts_tablet_manager.cc:1428] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:51:58.540634 12380 consensus_queue.cc:260] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.540870 12341 heartbeater.cc:503] Master 127.9.215.62:39643 was elected leader, sending a full tablet report...
I20260811 18:51:58.540934 12380 ts_tablet_manager.cc:1428] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:51:58.541081 12055 heartbeater.cc:503] Master 127.9.215.62:39643 was elected leader, sending a full tablet report...
I20260811 18:51:58.541049 12377 raft_consensus.cc:348] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.541271 12377 raft_consensus.cc:374] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:58.541325 12377 raft_consensus.cc:729] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f24ba3ddb524c6c86407366abeadfa3, State: Initialized, Role: FOLLOWER
I20260811 18:51:58.541442 12377 consensus_queue.cc:260] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.541740 12377 ts_tablet_manager.cc:1428] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 18:51:58.541792 11799 heartbeater.cc:503] Master 127.9.215.62:39643 was elected leader, sending a full tablet report...
W20260811 18:51:58.562752 12056 tablet.cc:2369] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:51:58.629226 11800 tablet.cc:2369] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:51:58.739630 12392 raft_consensus.cc:482] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:58.739794 12392 raft_consensus.cc:504] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.740171 12392 leader_election.cc:288] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929b3c449956469d9680d0cfb34df6f5 (127.9.215.2:40055), dbe7d0bab60049d4bc1c62bf598dc108 (127.9.215.3:45867)
I20260811 18:51:58.743408 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.740576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:42459 (local address 127.9.215.3:45867)
0811 18:51:58.740641 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.740643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.740804 (+ 161us) server_negotiation.cc:407] Connection header received
0811 18:51:58.740875 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.740876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.740891 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.740936 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:58.741100 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.741566 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.742680 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.743072 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.743134 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.743146 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.743296 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.743335 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:51:58.743423 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.740573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:37853 (local address 127.9.215.2:40055)
0811 18:51:58.740631 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.740634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.740798 (+ 164us) server_negotiation.cc:407] Connection header received
0811 18:51:58.740854 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.740855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.740869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.740918 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:58.741091 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.741600 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.742683 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.743069 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.743132 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.743145 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.743307 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.743348 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:51:58.743503 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.740430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:40055 (local address 127.9.215.1:37853)
0811 18:51:58.740704 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.740714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.740793 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.740936 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.740937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.740939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:58.741024 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.741026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.741685 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.741709 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.742578 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.742580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.743156 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.743157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.743233 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.743237 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.743285 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.743333 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":24,"thread_start_us":78,"threads_started":1}
I20260811 18:51:58.743512 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.740430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:45867 (local address 127.9.215.1:42459)
0811 18:51:58.740619 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.740632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:58.740798 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:58.740950 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:58.740951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:58.740953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:58.741036 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.741038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.741669 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.741670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.742578 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:58.742580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.743155 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.743156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:58.743223 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.743227 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:51:58.743280 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 18:51:58.743328 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":49,"thread_start_us":124,"threads_started":1}
I20260811 18:51:58.743932 12239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb63954523146e58f22f4e404a96cd2" candidate_uuid: "2f24ba3ddb524c6c86407366abeadfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" is_pre_election: true
I20260811 18:51:58.743932 11969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb63954523146e58f22f4e404a96cd2" candidate_uuid: "2f24ba3ddb524c6c86407366abeadfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929b3c449956469d9680d0cfb34df6f5" is_pre_election: true
I20260811 18:51:58.744112 11969 raft_consensus.cc:2454] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f24ba3ddb524c6c86407366abeadfa3 in term 0.
I20260811 18:51:58.744112 12239 raft_consensus.cc:2454] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f24ba3ddb524c6c86407366abeadfa3 in term 0.
I20260811 18:51:58.744366 11548 leader_election.cc:302] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2f24ba3ddb524c6c86407366abeadfa3, 929b3c449956469d9680d0cfb34df6f5, dbe7d0bab60049d4bc1c62bf598dc108; no voters:
I20260811 18:51:58.744551 12392 raft_consensus.cc:2781] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:51:58.744597 12392 raft_consensus.cc:482] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:51:58.744612 12392 raft_consensus.cc:3037] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:58.745334 12392 raft_consensus.cc:504] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
I20260811 18:51:58.745476 12392 leader_election.cc:288] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [CANDIDATE]: Term 1 election: Requested vote from peers 929b3c449956469d9680d0cfb34df6f5 (127.9.215.2:40055), dbe7d0bab60049d4bc1c62bf598dc108 (127.9.215.3:45867)
I20260811 18:51:58.745695 12239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb63954523146e58f22f4e404a96cd2" candidate_uuid: "2f24ba3ddb524c6c86407366abeadfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe7d0bab60049d4bc1c62bf598dc108"
I20260811 18:51:58.745790 12239 raft_consensus.cc:3037] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:58.746677 12239 raft_consensus.cc:2454] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f24ba3ddb524c6c86407366abeadfa3 in term 1.
I20260811 18:51:58.746809 11969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb63954523146e58f22f4e404a96cd2" candidate_uuid: "2f24ba3ddb524c6c86407366abeadfa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929b3c449956469d9680d0cfb34df6f5"
I20260811 18:51:58.746893 11543 leader_election.cc:302] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f24ba3ddb524c6c86407366abeadfa3, dbe7d0bab60049d4bc1c62bf598dc108; no voters:
I20260811 18:51:58.746907 11969 raft_consensus.cc:3037] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:58.746994 12392 raft_consensus.cc:2781] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:58.747156 12392 raft_consensus.cc:686] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [term 1 LEADER]: Becoming Leader. State: Replica: 2f24ba3ddb524c6c86407366abeadfa3, State: Running, Role: LEADER
I20260811 18:51:58.747277 12392 consensus_queue.cc:237] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } }
W20260811 18:51:58.747857 12342 tablet.cc:2369] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:51:58.747906 11969 raft_consensus.cc:2454] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f24ba3ddb524c6c86407366abeadfa3 in term 1.
I20260811 18:51:58.748222 11370 catalog_manager.cc:5495] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f24ba3ddb524c6c86407366abeadfa3 (127.9.215.1). New cstate: current_term: 1 leader_uuid: "2f24ba3ddb524c6c86407366abeadfa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f24ba3ddb524c6c86407366abeadfa3" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 } health_report { overall_health: UNKNOWN } } }
I20260811 18:51:58.758066 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.755731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.9.215.1:42267)
0811 18:51:58.755797 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:58.755799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:58.755808 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:51:58.755852 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:58.755853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:58.755878 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:58.755939 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:51:58.756071 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.756568 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.757332 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:58.757533 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:58.757587 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:58.757752 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:51:58.757865 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:51:58.757940 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:58.757959 (+ 19us) server_negotiation.cc:299] Negotiation successful
0811 18:51:58.757983 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:51:58.760051 11969 raft_consensus.cc:1261] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 1 FOLLOWER]: Refusing update from remote peer 2f24ba3ddb524c6c86407366abeadfa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:58.760052 12239 raft_consensus.cc:1261] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 1 FOLLOWER]: Refusing update from remote peer 2f24ba3ddb524c6c86407366abeadfa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:58.760438 12399 consensus_queue.cc:1035] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe7d0bab60049d4bc1c62bf598dc108" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 45867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:58.760586 12392 consensus_queue.cc:1035] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929b3c449956469d9680d0cfb34df6f5" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 40055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:58.763398 12407 mvcc.cc:205] Tried to move back new op lower bound from 7317398809638084608 to 7317398809589473280. Current Snapshot: MvccSnapshot[applied={T|T < 7317398809638084608}]
I20260811 18:51:58.763643 12406 mvcc.cc:205] Tried to move back new op lower bound from 7317398809638084608 to 7317398809589473280. Current Snapshot: MvccSnapshot[applied={T|T < 7317398809638084608}]
I20260811 18:51:58.765640 11368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:58.765762 11368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:58.766544 11368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aea322dd507a4e839b07c7cfb0d47545: Sending DeleteTablet for 3 replicas of tablet 3fb63954523146e58f22f4e404a96cd2
I20260811 18:51:58.766883 11649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fb63954523146e58f22f4e404a96cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:58 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:52230
I20260811 18:51:58.766939 11934 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fb63954523146e58f22f4e404a96cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:58 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:50192
I20260811 18:51:58.767159 12204 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fb63954523146e58f22f4e404a96cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:58 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:58028
I20260811 18:51:58.767414 12425 tablet_replica.cc:330] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: stopping tablet replica
I20260811 18:51:58.767438 12421 tablet_replica.cc:330] T 3fb63954523146e58f22f4e404a96cd2 P 2f24ba3ddb524c6c86407366abeadfa3: stopping tablet replica
I20260811 18:51:58.767622 12423 tablet_replica.cc:330] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: stopping tablet replica
I20260811 18:51:58.767620 12425 raft_consensus.cc:2229] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:58.767632 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11508
I20260811 18:51:58.767763 12425 raft_consensus.cc:2258] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:58.771703 12425 ts_tablet_manager.cc:1905] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:58.773465 12425 ts_tablet_manager.cc:1918] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:58.773554 12425 log.cc:1196] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/3fb63954523146e58f22f4e404a96cd2
I20260811 18:51:58.773797 12425 ts_tablet_manager.cc:1939] T 3fb63954523146e58f22f4e404a96cd2 P dbe7d0bab60049d4bc1c62bf598dc108: Deleting consensus metadata
I20260811 18:51:58.774328 11341 catalog_manager.cc:4841] TS dbe7d0bab60049d4bc1c62bf598dc108 (127.9.215.3:45867): tablet 3fb63954523146e58f22f4e404a96cd2 (table test-table [id=e1b074b5114646f18277e60752bf8d2d]) successfully deleted
I20260811 18:51:58.781594 12423 raft_consensus.cc:2229] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:58.781833 12423 raft_consensus.cc:2258] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:58.800793 12423 ts_tablet_manager.cc:1905] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:58.818565 12423 ts_tablet_manager.cc:1918] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:58.818748 12423 log.cc:1196] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/3fb63954523146e58f22f4e404a96cd2
I20260811 18:51:58.819053 12423 ts_tablet_manager.cc:1939] T 3fb63954523146e58f22f4e404a96cd2 P 929b3c449956469d9680d0cfb34df6f5: Deleting consensus metadata
I20260811 18:51:58.819667 11345 catalog_manager.cc:4841] TS 929b3c449956469d9680d0cfb34df6f5 (127.9.215.2:40055): tablet 3fb63954523146e58f22f4e404a96cd2 (table test-table [id=e1b074b5114646f18277e60752bf8d2d]) successfully deleted
W20260811 18:51:58.823314 11345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:42267 (error 108)
I20260811 18:51:58.823673 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11813
I20260811 18:51:58.829793 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:58.823822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42267 (local address 127.0.0.1:52248)
0811 18:51:58.829643 (+ 5821us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:58.829741 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:42267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5777}
W20260811 18:51:58.842936 11345 catalog_manager.cc:4584] TS 2f24ba3ddb524c6c86407366abeadfa3 (127.9.215.1:42267): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3fb63954523146e58f22f4e404a96cd2: Network error: Client connection negotiation failed: client connection to 127.9.215.1:42267: connect: Connection refused (error 111)
I20260811 18:51:58.849329 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12064
I20260811 18:51:58.876893 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 11311
2026-08-11T18:51:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2501 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T18:51:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:51:58Z Disabled control of system clock
I20260811 18:51:58.913116 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34099
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:34099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:51:59.000109 12432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:59.000367 12432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:59.000389 12432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:59.001667 12432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:51:59.001732 12432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:59.001744 12432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:59.001755 12432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:51:59.001768 12432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:51:59.003346 12432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:34099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:59.003698 12432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:59.004036 12432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:59.009547 12448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.009552 12443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.009547 12444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:59.009775 12432 server_base.cc:1034] running on GCE node
I20260811 18:51:59.010377 12432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:59.010742 12432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:59.011932 12432 hybrid_clock.cc:648] HybridClock initialized: now 1786474319011893 us; error 55 us; skew 500 ppm
I20260811 18:51:59.017277 12432 webserver.cc:460] Webserver started at http://127.9.215.62:36689/ using document root <none> and password file <none>
I20260811 18:51:59.017597 12432 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:59.017664 12432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:59.017827 12432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:59.019141 12432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "b34f26b8552443639e832133cbbd290d"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "4e44bb7c5f28d7082af629380d47dbea"
server_key_iv: "abedb29888eb43d37e607e4ce3e746ca"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.019606 12432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "b34f26b8552443639e832133cbbd290d"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "4e44bb7c5f28d7082af629380d47dbea"
server_key_iv: "abedb29888eb43d37e607e4ce3e746ca"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.021178 12432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:59.022166 12461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.022368 12432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:59.022475 12432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "b34f26b8552443639e832133cbbd290d"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "4e44bb7c5f28d7082af629380d47dbea"
server_key_iv: "abedb29888eb43d37e607e4ce3e746ca"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.022547 12432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:59.042377 12432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:59.042754 12432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:59.042906 12432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:59.047191 12432 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:34099
I20260811 18:51:59.047260 12567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:34099 every 8 connection(s)
I20260811 18:51:59.047663 12432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:51:59.048707 12568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:59.049690 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12432
I20260811 18:51:59.049800 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:51:59.049995 10076 external_mini_cluster.cc:1428] Setting key 646e91567502fd2200dc0312276df1c0
I20260811 18:51:59.051614 12568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d: Bootstrap starting.
I20260811 18:51:59.052338 12568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:59.052659 12568 log.cc:824] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:59.053411 12568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d: No bootstrap required, opened a new log
I20260811 18:51:59.055413 12568 raft_consensus.cc:348] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b34f26b8552443639e832133cbbd290d" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34099 } }
I20260811 18:51:59.055657 12568 raft_consensus.cc:374] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:59.055702 12568 raft_consensus.cc:729] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34f26b8552443639e832133cbbd290d, State: Initialized, Role: FOLLOWER
I20260811 18:51:59.055855 12568 consensus_queue.cc:260] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b34f26b8552443639e832133cbbd290d" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34099 } }
I20260811 18:51:59.055953 12568 raft_consensus.cc:388] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:51:59.055989 12568 raft_consensus.cc:482] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:51:59.056066 12568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:59.056344 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.050657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.9.215.62:34099)
0811 18:51:59.050858 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.050865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.050894 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:51:59.051371 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.051378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.051447 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.051630 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:59.051904 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.052957 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.054047 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.054556 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.054693 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.055408 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:59.055412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:59.055413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:51:59.055650 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:59.055688 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:51:59.055701 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:59.055774 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.056017 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.056129 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260811 18:51:59.056864 12568 raft_consensus.cc:504] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b34f26b8552443639e832133cbbd290d" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34099 } }
I20260811 18:51:59.056982 12568 leader_election.cc:302] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b34f26b8552443639e832133cbbd290d; no voters:
I20260811 18:51:59.057173 12568 leader_election.cc:288] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:51:59.057235 12579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:59.057353 12579 raft_consensus.cc:686] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [term 1 LEADER]: Becoming Leader. State: Replica: b34f26b8552443639e832133cbbd290d, State: Running, Role: LEADER
I20260811 18:51:59.057478 12568 sys_catalog.cc:564] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:51:59.057515 12579 consensus_queue.cc:237] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b34f26b8552443639e832133cbbd290d" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34099 } }
I20260811 18:51:59.057896 12583 sys_catalog.cc:455] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b34f26b8552443639e832133cbbd290d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b34f26b8552443639e832133cbbd290d" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34099 } } }
I20260811 18:51:59.057904 12584 sys_catalog.cc:455] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b34f26b8552443639e832133cbbd290d. Latest consensus state: current_term: 1 leader_uuid: "b34f26b8552443639e832133cbbd290d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b34f26b8552443639e832133cbbd290d" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34099 } } }
I20260811 18:51:59.058084 12584 sys_catalog.cc:458] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:59.058362 12583 sys_catalog.cc:458] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d [sys.catalog]: This master's current role is: LEADER
I20260811 18:51:59.058554 12596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:51:59.059043 12596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:51:59.060735 12596 catalog_manager.cc:1347] Generated new cluster ID: a4ba0e1197134d3f945332ffd371d8de
I20260811 18:51:59.060796 12596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:51:59.075678 12596 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:51:59.076254 12596 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:51:59.081331 12596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d: Generated new TSK 0
I20260811 18:51:59.081568 12596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:51:59.089221 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34099
--builtin_ntp_servers=127.9.215.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:51:59.179818 12618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:59.180060 12618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:59.180085 12618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:59.181641 12618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:59.181711 12618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:59.181751 12618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:51:59.183444 12618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:59.183759 12618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:59.184103 12618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:59.190326 12636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.190320 12630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:59.190522 12618 server_base.cc:1034] running on GCE node
W20260811 18:51:59.190754 12631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:59.190984 12618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:59.191256 12618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:59.192426 12618 hybrid_clock.cc:648] HybridClock initialized: now 1786474319192397 us; error 46 us; skew 500 ppm
I20260811 18:51:59.193817 12618 webserver.cc:460] Webserver started at http://127.9.215.1:45937/ using document root <none> and password file <none>
I20260811 18:51:59.194080 12618 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:59.194137 12618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:59.194280 12618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:59.195488 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "3e3d93294b44443185ed0ba4d1d049da"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "52c5715e877afe15a81959813d5702b0"
server_key_iv: "2761c234f2cac221870db73e96e1ddac"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.195891 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "3e3d93294b44443185ed0ba4d1d049da"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "52c5715e877afe15a81959813d5702b0"
server_key_iv: "2761c234f2cac221870db73e96e1ddac"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.197312 12618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:51:59.198210 12646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.198385 12618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:51:59.198479 12618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "3e3d93294b44443185ed0ba4d1d049da"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "52c5715e877afe15a81959813d5702b0"
server_key_iv: "2761c234f2cac221870db73e96e1ddac"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.198549 12618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:59.209146 12618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:59.209496 12618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:59.209635 12618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:59.209880 12618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:59.210435 12618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:59.210525 12618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.210574 12618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:59.210593 12618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.216874 12618 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:42617
I20260811 18:51:59.216918 12868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:42617 every 8 connection(s)
I20260811 18:51:59.217314 12618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:51:59.223047 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.218175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:58601 (local address 127.9.215.62:34099)
0811 18:51:59.218311 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.218313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.218476 (+ 163us) server_negotiation.cc:407] Connection header received
0811 18:51:59.219044 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.219046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.219061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.219110 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:59.219891 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.220424 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.221536 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.221749 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.221815 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.222754 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:59.222756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:59.222756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:51:59.222762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:59.222779 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:51:59.222783 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:59.222836 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.222946 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.222977 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:51:59.223214 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.218106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34099 (local address 127.9.215.1:58601)
0811 18:51:59.218346 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.218384 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.218909 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.219210 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.219214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.219224 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:59.219782 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.219787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.220553 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.220554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.221399 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.221402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.221845 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.221846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.221981 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.222588 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:51:59.222589 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:59.222624 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:51:59.222651 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:59.222846 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:59.222851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:59.222853 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.222907 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.223007 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260811 18:51:59.223785 12869 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34099
I20260811 18:51:59.223906 12869 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:59.224116 12869 heartbeater.cc:511] Master 127.9.215.62:34099 requested a full tablet report, sending...
I20260811 18:51:59.224851 12491 ts_manager.cc:194] Registered new tserver with Master: 3e3d93294b44443185ed0ba4d1d049da (127.9.215.1:42617)
I20260811 18:51:59.225641 12491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.1:58601
I20260811 18:51:59.225879 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12618
I20260811 18:51:59.225993 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:51:59.226212 10076 external_mini_cluster.cc:1428] Setting key 78ef5b74ad50d43f823373ab177d289a
I20260811 18:51:59.227201 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34099
--builtin_ntp_servers=127.9.215.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:51:59.314988 12876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:59.315238 12876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:59.315263 12876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:59.316629 12876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:59.316695 12876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:59.316725 12876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:51:59.318295 12876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:59.318590 12876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:59.318934 12876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:59.324461 12884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.324443 12882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.324450 12886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:59.324750 12876 server_base.cc:1034] running on GCE node
I20260811 18:51:59.325062 12876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:59.325349 12876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:59.326535 12876 hybrid_clock.cc:648] HybridClock initialized: now 1786474319326513 us; error 32 us; skew 500 ppm
I20260811 18:51:59.328244 12876 webserver.cc:460] Webserver started at http://127.9.215.2:46599/ using document root <none> and password file <none>
I20260811 18:51:59.328524 12876 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:59.328583 12876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:59.328836 12876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:59.330159 12876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "d5100bd515f44458aefae41beb393534"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "c9d8c1de907538611e59e3ff43a0ab6b"
server_key_iv: "8924ac50c4d2ffff47bbcccac75238f9"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.330611 12876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "d5100bd515f44458aefae41beb393534"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "c9d8c1de907538611e59e3ff43a0ab6b"
server_key_iv: "8924ac50c4d2ffff47bbcccac75238f9"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.332254 12876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 18:51:59.333390 12899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.333637 12876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 18:51:59.333755 12876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "d5100bd515f44458aefae41beb393534"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "c9d8c1de907538611e59e3ff43a0ab6b"
server_key_iv: "8924ac50c4d2ffff47bbcccac75238f9"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.333837 12876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:59.343498 12876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:59.343896 12876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:59.344051 12876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:59.344314 12876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:59.344677 12876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:59.344715 12876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.344758 12876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:59.344781 12876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.351393 12876 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42481
I20260811 18:51:59.351495 13115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42481 every 8 connection(s)
I20260811 18:51:59.351847 12876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:51:59.353540 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 12876
I20260811 18:51:59.353755 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:51:59.353969 10076 external_mini_cluster.cc:1428] Setting key e3f2ebf4ba5f124b3473c9d5698a8141
I20260811 18:51:59.354907 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34099
--builtin_ntp_servers=127.9.215.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:51:59.357592 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.352628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:39163 (local address 127.9.215.62:34099)
0811 18:51:59.352683 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.352685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.352915 (+ 230us) server_negotiation.cc:407] Connection header received
0811 18:51:59.353490 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.353492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.353506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.353558 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:59.354358 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.354867 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.356059 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.356257 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.356317 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.357293 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:59.357295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:59.357296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:51:59.357302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:59.357317 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:51:59.357322 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:59.357366 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.357472 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.357505 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:51:59.357873 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.352603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34099 (local address 127.9.215.2:39163)
0811 18:51:59.352788 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.352827 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.353375 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.353637 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.353643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.353653 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:59.354264 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.354270 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.354965 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.354966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.355959 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.355962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.356341 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.356342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.356493 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.357117 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:51:59.357117 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:59.357152 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:51:59.357180 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:59.357384 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:59.357391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:59.357394 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.357464 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.357585 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260811 18:51:59.358358 13116 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34099
I20260811 18:51:59.358469 13116 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:59.358675 13116 heartbeater.cc:511] Master 127.9.215.62:34099 requested a full tablet report, sending...
I20260811 18:51:59.359061 12491 ts_manager.cc:194] Registered new tserver with Master: d5100bd515f44458aefae41beb393534 (127.9.215.2:42481)
I20260811 18:51:59.359524 12491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.2:39163
W20260811 18:51:59.441998 13120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:51:59.442231 13120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:51:59.442255 13120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:51:59.443534 13120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:51:59.443588 13120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:51:59.443620 13120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:51:59.445164 13120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:51:59.445416 13120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:51:59.445780 13120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:51:59.451282 13129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.451297 13128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:51:59.451284 13141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:51:59.451485 13120 server_base.cc:1034] running on GCE node
I20260811 18:51:59.451815 13120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:51:59.452088 13120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:51:59.453279 13120 hybrid_clock.cc:648] HybridClock initialized: now 1786474319453263 us; error 33 us; skew 500 ppm
I20260811 18:51:59.454836 13120 webserver.cc:460] Webserver started at http://127.9.215.3:43419/ using document root <none> and password file <none>
I20260811 18:51:59.455117 13120 fs_manager.cc:359] Metadata directory not provided
I20260811 18:51:59.455169 13120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:51:59.455286 13120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:51:59.456503 13120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "8b9d9bd0d9e9478d864b57521e04bb04"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "74a998b0c2f786f91d5758e313f4197a"
server_key_iv: "06b3bb686b2d709e22849d166a0d1b9b"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.456857 13120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "8b9d9bd0d9e9478d864b57521e04bb04"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "74a998b0c2f786f91d5758e313f4197a"
server_key_iv: "06b3bb686b2d709e22849d166a0d1b9b"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.458299 13120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:51:59.459163 13150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.459350 13120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:51:59.459450 13120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "8b9d9bd0d9e9478d864b57521e04bb04"
format_stamp: "Formatted at 2026-08-11 18:51:59 on dist-test-slave-pww0"
server_key: "74a998b0c2f786f91d5758e313f4197a"
server_key_iv: "06b3bb686b2d709e22849d166a0d1b9b"
server_key_version: "encryptionkey@0"
I20260811 18:51:59.459559 13120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:51:59.475323 13120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:51:59.475729 13120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:51:59.475883 13120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:51:59.476130 13120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:51:59.476481 13120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:51:59.476526 13120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.476557 13120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:51:59.476574 13120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:51:59.482456 13120 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:40555
I20260811 18:51:59.482530 13379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:40555 every 8 connection(s)
I20260811 18:51:59.482865 13120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:51:59.488468 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.483693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:36883 (local address 127.9.215.62:34099)
0811 18:51:59.483737 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.483739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.483981 (+ 242us) server_negotiation.cc:407] Connection header received
0811 18:51:59.484482 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.484483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.484494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.484546 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:59.485199 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.485692 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.486777 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.486986 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.487093 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.488165 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:59.488166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:59.488167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:51:59.488173 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:59.488191 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:51:59.488195 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:59.488244 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.488356 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.488387 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:51:59.488615 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.483656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34099 (local address 127.9.215.3:36883)
0811 18:51:59.483850 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.483888 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.484389 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.484610 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.484614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.484623 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:51:59.485125 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.485130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.485776 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.485777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.486642 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.486645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.487155 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.487156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.487303 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.487970 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:51:59.487971 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:51:59.488005 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:51:59.488033 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:51:59.488252 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:51:59.488255 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:51:59.488258 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.488308 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.488411 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260811 18:51:59.489233 13380 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34099
I20260811 18:51:59.489342 13380 heartbeater.cc:464] Registering TS with master...
I20260811 18:51:59.489552 13380 heartbeater.cc:511] Master 127.9.215.62:34099 requested a full tablet report, sending...
I20260811 18:51:59.490010 12491 ts_manager.cc:194] Registered new tserver with Master: 8b9d9bd0d9e9478d864b57521e04bb04 (127.9.215.3:40555)
I20260811 18:51:59.490413 12491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.3:36883
I20260811 18:51:59.492036 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13120
I20260811 18:51:59.492142 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:51:59.492319 10076 external_mini_cluster.cc:1428] Setting key 5e83b29ae8ddacd3377d72c939de3350
I20260811 18:51:59.493883 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:51:59.498667 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.495293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.9.215.62:34099)
0811 18:51:59.495354 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.495356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.495495 (+ 139us) server_negotiation.cc:407] Connection header received
0811 18:51:59.495530 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.495531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.495544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.495614 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:51:59.495862 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.496469 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.497354 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.497559 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.497622 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.498453 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:51:59.498455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:51:59.498455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:51:59.498461 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:51:59.498478 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:51:59.498482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:51:59.498528 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.498574 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.498599 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:51:59.500440 12491 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:51:59.502136 12491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:51:59.509450 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.504799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.9.215.2:42481)
0811 18:51:59.504948 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.504953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.505138 (+ 185us) server_negotiation.cc:407] Connection header received
0811 18:51:59.505188 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.505191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.505207 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.505281 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:59.505649 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.506553 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.508189 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.508963 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.509043 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.509147 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.509313 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.509355 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260811 18:51:59.509472 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.504665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42617 (local address 127.0.0.1:60050)
0811 18:51:59.504870 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.504916 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.504991 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.505690 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.505691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.505693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:59.505748 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.505749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.507021 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.507022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.508072 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.508074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.509066 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.509066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.509182 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.509202 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.509285 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.509362 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":29,"thread_start_us":92,"threads_started":1}
I20260811 18:51:59.509785 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.505090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.9.215.1:42617)
0811 18:51:59.505285 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.505290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.505300 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:51:59.505524 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.505531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.505552 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.505689 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:59.505822 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.506925 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.508160 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.508962 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.509043 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.509147 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.509642 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.509689 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260811 18:51:59.509472 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.504682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42481 (local address 127.0.0.1:49870)
0811 18:51:59.505056 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.505065 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.505122 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.505429 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.505433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.505435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:59.505513 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.505515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.506636 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.506636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.508068 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.508071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.509064 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.509065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.509182 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.509202 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.509285 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.509362 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":21,"thread_start_us":113,"threads_started":1}
I20260811 18:51:59.510659 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.504908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:40555 (local address 127.0.0.1:45586)
0811 18:51:59.505748 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.505759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.505810 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.506011 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.506012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.506014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:59.506051 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.506052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.507221 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.507222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.508069 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.508072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.510415 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.510416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.510485 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.510490 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.510546 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.510590 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":103,"threads_started":1}
I20260811 18:51:59.510677 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.505447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.9.215.3:40555)
0811 18:51:59.505694 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.505698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.505863 (+ 165us) server_negotiation.cc:407] Connection header received
0811 18:51:59.505909 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.505911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.505926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.506001 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:59.506108 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.507133 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.509192 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.510326 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.510405 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.510500 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.510558 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.510600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260811 18:51:59.510546 12740 tablet_service.cc:1467] Processing CreateTablet for tablet b19a7c6030c04d8985aa93baafab9831 (DEFAULT_TABLE table=test-table [id=36b723e92fe04d36b35ef0be761e1cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:59.510852 12995 tablet_service.cc:1467] Processing CreateTablet for tablet b19a7c6030c04d8985aa93baafab9831 (DEFAULT_TABLE table=test-table [id=36b723e92fe04d36b35ef0be761e1cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:59.510984 12740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b19a7c6030c04d8985aa93baafab9831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:59.511202 12995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b19a7c6030c04d8985aa93baafab9831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:59.511366 13240 tablet_service.cc:1467] Processing CreateTablet for tablet b19a7c6030c04d8985aa93baafab9831 (DEFAULT_TABLE table=test-table [id=36b723e92fe04d36b35ef0be761e1cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:51:59.511718 13240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b19a7c6030c04d8985aa93baafab9831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:51:59.513746 13403 tablet_bootstrap.cc:492] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: Bootstrap starting.
I20260811 18:51:59.514447 13403 tablet_bootstrap.cc:654] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:59.514600 13407 tablet_bootstrap.cc:492] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Bootstrap starting.
I20260811 18:51:59.514770 13403 log.cc:824] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:59.515264 13407 tablet_bootstrap.cc:654] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:59.515308 13408 tablet_bootstrap.cc:492] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Bootstrap starting.
I20260811 18:51:59.515627 13407 log.cc:824] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:59.515765 13403 tablet_bootstrap.cc:492] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: No bootstrap required, opened a new log
I20260811 18:51:59.515841 13403 ts_tablet_manager.cc:1397] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:51:59.516465 13407 tablet_bootstrap.cc:492] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: No bootstrap required, opened a new log
I20260811 18:51:59.516548 13407 ts_tablet_manager.cc:1397] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:51:59.518618 13403 raft_consensus.cc:348] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.518831 13403 raft_consensus.cc:374] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:59.518874 13403 raft_consensus.cc:729] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3d93294b44443185ed0ba4d1d049da, State: Initialized, Role: FOLLOWER
I20260811 18:51:59.519008 13403 consensus_queue.cc:260] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.519351 13403 ts_tablet_manager.cc:1428] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 18:51:59.519258 13407 raft_consensus.cc:348] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.519397 12869 heartbeater.cc:503] Master 127.9.215.62:34099 was elected leader, sending a full tablet report...
I20260811 18:51:59.519420 13407 raft_consensus.cc:374] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:59.519483 13407 raft_consensus.cc:729] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5100bd515f44458aefae41beb393534, State: Initialized, Role: FOLLOWER
I20260811 18:51:59.519639 13407 consensus_queue.cc:260] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.519943 13407 ts_tablet_manager.cc:1428] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 18:51:59.520067 13116 heartbeater.cc:503] Master 127.9.215.62:34099 was elected leader, sending a full tablet report...
I20260811 18:51:59.520298 13408 tablet_bootstrap.cc:654] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Neither blocks nor log segments found. Creating new log.
I20260811 18:51:59.520659 13408 log.cc:824] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Log is configured to *not* fsync() on all Append() calls
I20260811 18:51:59.521463 13408 tablet_bootstrap.cc:492] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: No bootstrap required, opened a new log
I20260811 18:51:59.521528 13408 ts_tablet_manager.cc:1397] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 18:51:59.523381 13408 raft_consensus.cc:348] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.523598 13408 raft_consensus.cc:374] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:51:59.523628 13408 raft_consensus.cc:729] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9d9bd0d9e9478d864b57521e04bb04, State: Initialized, Role: FOLLOWER
I20260811 18:51:59.523734 13408 consensus_queue.cc:260] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.523998 13408 ts_tablet_manager.cc:1428] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:51:59.524097 13380 heartbeater.cc:503] Master 127.9.215.62:34099 was elected leader, sending a full tablet report...
I20260811 18:51:59.600492 13413 raft_consensus.cc:482] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:51:59.600644 13413 raft_consensus.cc:504] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.600977 13413 leader_election.cc:288] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b9d9bd0d9e9478d864b57521e04bb04 (127.9.215.3:40555), d5100bd515f44458aefae41beb393534 (127.9.215.2:42481)
W20260811 18:51:59.602468 13117 tablet.cc:2369] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:51:59.604235 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.601250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:40075 (local address 127.9.215.2:42481)
0811 18:51:59.601294 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.601297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.601309 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:51:59.601347 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.601348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.601359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.601440 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:59.601679 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.602268 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.603440 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.603867 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.603934 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.603948 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.604093 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.604139 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:51:59.604243 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.601248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:46715 (local address 127.9.215.3:40555)
0811 18:51:59.601305 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.601307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.601516 (+ 209us) server_negotiation.cc:407] Connection header received
0811 18:51:59.601553 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.601554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.601565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.601621 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:51:59.601851 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.602427 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.603491 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.603883 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.603946 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.603959 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.604126 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.604173 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:51:59.604235 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.601176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42481 (local address 127.9.215.1:40075)
0811 18:51:59.601241 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.601253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.601297 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.601482 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.601484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.601486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:59.601571 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.601573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.602378 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.602379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.603345 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.603349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.603958 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.603960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.604027 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.604030 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.604078 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.604129 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260811 18:51:59.604249 13418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.601187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:40555 (local address 127.9.215.1:46715)
0811 18:51:59.601388 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.601398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:51:59.601467 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:51:59.601678 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:51:59.601679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:51:59.601681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:51:59.601768 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.601770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.602508 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.602509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.603371 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:51:59.603373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.603969 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.603971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:51:59.604055 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.604058 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:51:59.604104 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 18:51:59.604148 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":22,"thread_start_us":114,"threads_started":1}
I20260811 18:51:59.604676 13026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b19a7c6030c04d8985aa93baafab9831" candidate_uuid: "3e3d93294b44443185ed0ba4d1d049da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5100bd515f44458aefae41beb393534" is_pre_election: true
I20260811 18:51:59.604857 13026 raft_consensus.cc:2454] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3d93294b44443185ed0ba4d1d049da in term 0.
I20260811 18:51:59.604871 13271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b19a7c6030c04d8985aa93baafab9831" candidate_uuid: "3e3d93294b44443185ed0ba4d1d049da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" is_pre_election: true
I20260811 18:51:59.605041 13271 raft_consensus.cc:2454] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e3d93294b44443185ed0ba4d1d049da in term 0.
I20260811 18:51:59.605108 12650 leader_election.cc:302] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3d93294b44443185ed0ba4d1d049da, d5100bd515f44458aefae41beb393534; no voters:
I20260811 18:51:59.605266 13413 raft_consensus.cc:2781] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:51:59.605336 13413 raft_consensus.cc:482] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:51:59.605360 13413 raft_consensus.cc:3037] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:59.606140 13413 raft_consensus.cc:504] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.606295 13413 leader_election.cc:288] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [CANDIDATE]: Term 1 election: Requested vote from peers 8b9d9bd0d9e9478d864b57521e04bb04 (127.9.215.3:40555), d5100bd515f44458aefae41beb393534 (127.9.215.2:42481)
I20260811 18:51:59.606478 13271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b19a7c6030c04d8985aa93baafab9831" candidate_uuid: "3e3d93294b44443185ed0ba4d1d049da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9d9bd0d9e9478d864b57521e04bb04"
I20260811 18:51:59.606539 13026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b19a7c6030c04d8985aa93baafab9831" candidate_uuid: "3e3d93294b44443185ed0ba4d1d049da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5100bd515f44458aefae41beb393534"
I20260811 18:51:59.606580 13271 raft_consensus.cc:3037] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:59.606626 13026 raft_consensus.cc:3037] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:51:59.607575 13271 raft_consensus.cc:2454] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3d93294b44443185ed0ba4d1d049da in term 1.
I20260811 18:51:59.607575 13026 raft_consensus.cc:2454] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e3d93294b44443185ed0ba4d1d049da in term 1.
I20260811 18:51:59.607818 12648 leader_election.cc:302] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3d93294b44443185ed0ba4d1d049da, 8b9d9bd0d9e9478d864b57521e04bb04; no voters:
I20260811 18:51:59.607916 13413 raft_consensus.cc:2781] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:51:59.608098 13413 raft_consensus.cc:686] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 1 LEADER]: Becoming Leader. State: Replica: 3e3d93294b44443185ed0ba4d1d049da, State: Running, Role: LEADER
I20260811 18:51:59.608206 13413 consensus_queue.cc:237] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } }
I20260811 18:51:59.609316 12491 catalog_manager.cc:5495] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e3d93294b44443185ed0ba4d1d049da (127.9.215.1). New cstate: current_term: 1 leader_uuid: "3e3d93294b44443185ed0ba4d1d049da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3d93294b44443185ed0ba4d1d049da" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 } health_report { overall_health: UNKNOWN } } }
I20260811 18:51:59.626669 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.624241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.9.215.1:42617)
0811 18:51:59.624295 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:51:59.624297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:51:59.624306 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:51:59.624373 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:51:59.624374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:51:59.624397 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:51:59.624462 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:51:59.624558 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.625042 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.625856 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:51:59.626065 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:51:59.626145 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:51:59.626279 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:51:59.626401 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:51:59.626479 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:51:59.626500 (+ 21us) server_negotiation.cc:299] Negotiation successful
0811 18:51:59.626544 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:51:59.628800 13026 raft_consensus.cc:1261] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 1 FOLLOWER]: Refusing update from remote peer 3e3d93294b44443185ed0ba4d1d049da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:59.628800 13271 raft_consensus.cc:1261] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 1 FOLLOWER]: Refusing update from remote peer 3e3d93294b44443185ed0ba4d1d049da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:51:59.629221 13420 consensus_queue.cc:1035] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b9d9bd0d9e9478d864b57521e04bb04" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:59.629361 13413 consensus_queue.cc:1035] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5100bd515f44458aefae41beb393534" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:51:59.633836 13433 mvcc.cc:205] Tried to move back new op lower bound from 7317398813196365824 to 7317398813116035072. Current Snapshot: MvccSnapshot[applied={T|T < 7317398813196365824 or (T in {7317398813196365824})}]
I20260811 18:51:59.634688 12491 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:59.634779 12491 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:51:59.635527 12491 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b34f26b8552443639e832133cbbd290d: Sending DeleteTablet for 3 replicas of tablet b19a7c6030c04d8985aa93baafab9831
I20260811 18:51:59.635864 12995 tablet_service.cc:1514] Processing DeleteTablet for tablet b19a7c6030c04d8985aa93baafab9831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:59 UTC) from {username='slave'} at 127.0.0.1:49870
I20260811 18:51:59.635951 12740 tablet_service.cc:1514] Processing DeleteTablet for tablet b19a7c6030c04d8985aa93baafab9831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:59 UTC) from {username='slave'} at 127.0.0.1:60050
I20260811 18:51:59.636008 13240 tablet_service.cc:1514] Processing DeleteTablet for tablet b19a7c6030c04d8985aa93baafab9831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:51:59 UTC) from {username='slave'} at 127.0.0.1:45586
I20260811 18:51:59.636161 13453 tablet_replica.cc:330] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: stopping tablet replica
I20260811 18:51:59.636266 13453 raft_consensus.cc:2229] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:59.636364 13453 raft_consensus.cc:2258] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:59.636514 13454 tablet_replica.cc:330] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da: stopping tablet replica
I20260811 18:51:59.636620 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12618
I20260811 18:51:59.636662 13454 raft_consensus.cc:2229] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:51:59.636765 13454 raft_consensus.cc:2258] T b19a7c6030c04d8985aa93baafab9831 P 3e3d93294b44443185ed0ba4d1d049da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:59.640358 13453 ts_tablet_manager.cc:1905] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:59.643225 13453 ts_tablet_manager.cc:1918] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:59.643306 13453 log.cc:1196] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/b19a7c6030c04d8985aa93baafab9831
I20260811 18:51:59.643567 13453 ts_tablet_manager.cc:1939] T b19a7c6030c04d8985aa93baafab9831 P d5100bd515f44458aefae41beb393534: Deleting consensus metadata
I20260811 18:51:59.644085 12464 catalog_manager.cc:4841] TS d5100bd515f44458aefae41beb393534 (127.9.215.2:42481): tablet b19a7c6030c04d8985aa93baafab9831 (table test-table [id=36b723e92fe04d36b35ef0be761e1cf1]) successfully deleted
I20260811 18:51:59.670279 13461 tablet_replica.cc:330] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: stopping tablet replica
I20260811 18:51:59.670612 13461 raft_consensus.cc:2229] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:51:59.670849 13461 raft_consensus.cc:2258] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:51:59.685379 13461 ts_tablet_manager.cc:1905] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:51:59.686964 13461 ts_tablet_manager.cc:1918] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:51:59.687065 13461 log.cc:1196] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/b19a7c6030c04d8985aa93baafab9831
I20260811 18:51:59.687304 13461 ts_tablet_manager.cc:1939] T b19a7c6030c04d8985aa93baafab9831 P 8b9d9bd0d9e9478d864b57521e04bb04: Deleting consensus metadata
I20260811 18:51:59.687879 12463 catalog_manager.cc:4841] TS 8b9d9bd0d9e9478d864b57521e04bb04 (127.9.215.3:40555): tablet b19a7c6030c04d8985aa93baafab9831 (table test-table [id=36b723e92fe04d36b35ef0be761e1cf1]) successfully deleted
W20260811 18:51:59.689033 12463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:42617 (error 108)
I20260811 18:51:59.689253 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12876
I20260811 18:51:59.689550 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:51:59.689438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42617 (local address 127.0.0.1:60060)
0811 18:51:59.689485 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 18:51:59.689519 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:42617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 18:51:59.689657 12463 catalog_manager.cc:4584] TS 3e3d93294b44443185ed0ba4d1d049da (127.9.215.1:42617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b19a7c6030c04d8985aa93baafab9831: Network error: Client connection negotiation failed: client connection to 127.9.215.1:42617: connect: Connection refused (error 111)
I20260811 18:51:59.715339 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13120
I20260811 18:51:59.731004 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 12432
2026-08-11T18:51:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (867 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:51:59 dist-test-slave-pww0 krb5kdc[13465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:51:59 dist-test-slave-pww0 krb5kdc[13465](info): set up 2 sockets
Aug 11 18:51:59 dist-test-slave-pww0 krb5kdc[13465](info): commencing operation
krb5kdc: starting...
W20260811 18:52:00.976435 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:01.114212 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45323
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:45323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:01.206079 13495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:01.206343 13495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:01.206370 13495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:01.207690 13495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:01.207741 13495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:01.207755 13495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:01.207767 13495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:01.207778 13495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:01.209483 13495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:45323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:01.209745 13495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:01.209956 13495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:01.216190 13503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.216342 13500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.216190 13505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:01.216483 13495 server_base.cc:1034] running on GCE node
I20260811 18:52:01.216799 13495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:01.217195 13495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:01.218379 13495 hybrid_clock.cc:648] HybridClock initialized: now 1786474321218349 us; error 45 us; skew 500 ppm
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:01.221591 13495 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:01.222289 13495 webserver.cc:460] Webserver started at http://127.9.215.62:36817/ using document root <none> and password file <none>
I20260811 18:52:01.222544 13495 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:01.222601 13495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:01.222750 13495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:01.223964 13495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "e344e6cdd13b4be99cc747481643daf3"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "3ddac5fe1af5accb406490e6e16f4357"
server_key_iv: "7f0b03f241aab8866d1b7c872649fe1b"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.224331 13495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "e344e6cdd13b4be99cc747481643daf3"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "3ddac5fe1af5accb406490e6e16f4357"
server_key_iv: "7f0b03f241aab8866d1b7c872649fe1b"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.225879 13495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:01.226939 13514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.227223 13495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:01.227322 13495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "e344e6cdd13b4be99cc747481643daf3"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "3ddac5fe1af5accb406490e6e16f4357"
server_key_iv: "7f0b03f241aab8866d1b7c872649fe1b"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.227401 13495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:01.239295 13495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:01.241753 13495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:01.241935 13495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:01.246259 13495 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:45323
I20260811 18:52:01.246309 13627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:45323 every 8 connection(s)
I20260811 18:52:01.246690 13495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:01.247690 13628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:01.250337 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3: Bootstrap starting.
I20260811 18:52:01.250834 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13495
I20260811 18:52:01.250962 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:01.251065 13628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:01.251161 10076 external_mini_cluster.cc:1428] Setting key 17f0efd430df86e16a4ebacccb45697d
I20260811 18:52:01.251379 13628 log.cc:824] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:01.252280 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3: No bootstrap required, opened a new log
I20260811 18:52:01.254406 13628 raft_consensus.cc:348] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e344e6cdd13b4be99cc747481643daf3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45323 } }
I20260811 18:52:01.254590 13628 raft_consensus.cc:374] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:01.254612 13628 raft_consensus.cc:729] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e344e6cdd13b4be99cc747481643daf3, State: Initialized, Role: FOLLOWER
I20260811 18:52:01.254738 13628 consensus_queue.cc:260] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e344e6cdd13b4be99cc747481643daf3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45323 } }
I20260811 18:52:01.254822 13628 raft_consensus.cc:388] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:01.254853 13628 raft_consensus.cc:482] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:01.254896 13628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:01.255646 13628 raft_consensus.cc:504] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e344e6cdd13b4be99cc747481643daf3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45323 } }
I20260811 18:52:01.255882 13628 leader_election.cc:302] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e344e6cdd13b4be99cc747481643daf3; no voters:
I20260811 18:52:01.256167 13628 leader_election.cc:288] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:01.256194 13634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:01.256328 13634 raft_consensus.cc:686] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [term 1 LEADER]: Becoming Leader. State: Replica: e344e6cdd13b4be99cc747481643daf3, State: Running, Role: LEADER
I20260811 18:52:01.256500 13628 sys_catalog.cc:564] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:01.256500 13634 consensus_queue.cc:237] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e344e6cdd13b4be99cc747481643daf3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45323 } }
I20260811 18:52:01.257272 13635 sys_catalog.cc:455] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e344e6cdd13b4be99cc747481643daf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e344e6cdd13b4be99cc747481643daf3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45323 } } }
I20260811 18:52:01.257337 13639 sys_catalog.cc:455] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e344e6cdd13b4be99cc747481643daf3. Latest consensus state: current_term: 1 leader_uuid: "e344e6cdd13b4be99cc747481643daf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e344e6cdd13b4be99cc747481643daf3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45323 } } }
I20260811 18:52:01.257503 13639 sys_catalog.cc:458] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:01.257483 13635 sys_catalog.cc:458] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:01.257856 13650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:01.258621 13650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:01.260785 13650 catalog_manager.cc:1347] Generated new cluster ID: 21603c1eff014f128067f9a26068a3a9
I20260811 18:52:01.260851 13650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:01.271078 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.251769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.9.215.62:45323)
0811 18:52:01.251971 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.251980 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.252011 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:52:01.252570 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.252580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.252641 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.252794 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:01.253494 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.254683 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.255745 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.256336 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.257146 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.260215 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:01.260220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:01.260220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:01.260235 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:01.269296 (+ 9061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.269784 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.269788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.269789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.269902 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.270112 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.270113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.270114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.270420 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:01.270609 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.270697 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.270851 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260811 18:52:01.287770 13650 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:01.288581 13650 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:01.300029 13650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3: Generated new TSK 0
I20260811 18:52:01.300382 13650 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:52:01.346398 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45323
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:01.441730 13669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:01.441980 13669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:01.442006 13669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:01.443286 13669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:01.443346 13669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:01.443377 13669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:52:01.445075 13669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:01.445353 13669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:01.445611 13669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:01.451189 13685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.451238 13682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.451238 13683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:01.451452 13669 server_base.cc:1034] running on GCE node
I20260811 18:52:01.451731 13669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:01.451995 13669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:01.453176 13669 hybrid_clock.cc:648] HybridClock initialized: now 1786474321453150 us; error 41 us; skew 500 ppm
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:01.456346 13669 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:52:01.456995 13669 webserver.cc:460] Webserver started at http://127.9.215.1:35193/ using document root <none> and password file <none>
I20260811 18:52:01.457237 13669 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:01.457293 13669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:01.457418 13669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:01.458665 13669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "53f3a0a582ec4968b0de67544a26b795"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "779acf017b5a366be02a6b3017dcf33e"
server_key_iv: "1ebe82edf2f52003b5b341acd9b596df"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.459049 13669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "53f3a0a582ec4968b0de67544a26b795"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "779acf017b5a366be02a6b3017dcf33e"
server_key_iv: "1ebe82edf2f52003b5b341acd9b596df"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.460563 13669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:01.461426 13700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.461603 13669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:01.461714 13669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "53f3a0a582ec4968b0de67544a26b795"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "779acf017b5a366be02a6b3017dcf33e"
server_key_iv: "1ebe82edf2f52003b5b341acd9b596df"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.461791 13669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:01.470806 13669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:01.473570 13669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:01.473749 13669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:01.473995 13669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:01.474294 13669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:01.474327 13669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.474354 13669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:01.474370 13669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.480540 13669 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:43877
I20260811 18:52:01.480619 13918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:43877 every 8 connection(s)
I20260811 18:52:01.481050 13669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:01.484014 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13669
I20260811 18:52:01.484122 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:52:01.484323 10076 external_mini_cluster.cc:1428] Setting key 5db0e52b51701c41ca00411a3df6d914
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:01.491869 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.481934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:57271 (local address 127.9.215.62:45323)
0811 18:52:01.481988 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.481991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.482223 (+ 232us) server_negotiation.cc:407] Connection header received
0811 18:52:01.482781 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.482783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.482796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.482851 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:01.483923 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.484581 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.485611 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.485950 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.486039 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.488856 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:01.488858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:01.488858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:01.488867 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:01.490236 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.490742 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.490744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.490744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.490805 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.491069 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.491070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.491070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.491208 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:01.491273 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.491658 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.491770 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:52:01.492064 13921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.481857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45323 (local address 127.9.215.1:57271)
0811 18:52:01.482084 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:01.482129 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:01.482674 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:01.482931 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:01.482935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:01.483591 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:01.483818 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.483824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.484668 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.484669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.485514 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.485517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.486098 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.486100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.486235 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.486854 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:01.486866 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:01.488726 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:01.490357 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:01.490371 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:01.490375 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:01.490657 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:01.490869 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:01.490870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:01.490870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:01.491003 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:01.491275 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:01.491285 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:01.491449 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 18:52:01.491627 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 18:52:01.491822 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
I20260811 18:52:01.492662 13919 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45323
I20260811 18:52:01.492802 13919 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:01.493044 13919 heartbeater.cc:511] Master 127.9.215.62:45323 requested a full tablet report, sending...
I20260811 18:52:01.493765 13542 ts_manager.cc:194] Registered new tserver with Master: 53f3a0a582ec4968b0de67544a26b795 (127.9.215.1:43877)
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
I20260811 18:52:01.494426 13542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:57271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:52:01.515679 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45323
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:01.602432 13925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:01.602674 13925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:01.602711 13925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:01.604045 13925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:01.604103 13925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:01.604143 13925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:52:01.605790 13925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:01.606055 13925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:01.606318 13925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:01.612164 13932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.612170 13933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.612164 13936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:01.612874 13925 server_base.cc:1034] running on GCE node
I20260811 18:52:01.613143 13925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:01.613441 13925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:01.614648 13925 hybrid_clock.cc:648] HybridClock initialized: now 1786474321614618 us; error 37 us; skew 500 ppm
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:01.617530 13925 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:52:01.618062 13925 webserver.cc:460] Webserver started at http://127.9.215.2:46813/ using document root <none> and password file <none>
I20260811 18:52:01.618295 13925 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:01.618350 13925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:01.618484 13925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:01.619659 13925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "3c47ace89a7641359fefa16f0c3bc661"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "9bad16c6037b546c28d31a3627fe98e4"
server_key_iv: "089ad141c21bc5a3ffad6dbfca841a4f"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.620023 13925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "3c47ace89a7641359fefa16f0c3bc661"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "9bad16c6037b546c28d31a3627fe98e4"
server_key_iv: "089ad141c21bc5a3ffad6dbfca841a4f"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.621412 13925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:01.622273 13950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.622618 13925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:01.622781 13925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "3c47ace89a7641359fefa16f0c3bc661"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "9bad16c6037b546c28d31a3627fe98e4"
server_key_iv: "089ad141c21bc5a3ffad6dbfca841a4f"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.622874 13925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:01.656579 13925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:01.659124 13925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:01.659286 13925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:01.659559 13925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:01.659921 13925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:01.659960 13925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.659989 13925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:01.660006 13925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.665978 13925 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:42889
I20260811 18:52:01.666034 14170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:42889 every 8 connection(s)
I20260811 18:52:01.666374 13925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:01.672870 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 13925
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:01.673003 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:52:01.673213 10076 external_mini_cluster.cc:1428] Setting key b1873cec29517e4602f9301c0dd4b2ce
I20260811 18:52:01.676383 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.667204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:39029 (local address 127.9.215.62:45323)
0811 18:52:01.667302 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.667305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.667485 (+ 180us) server_negotiation.cc:407] Connection header received
0811 18:52:01.668117 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.668119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.668133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.668197 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:01.669232 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.669751 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.670828 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.671043 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.671109 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.673457 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:01.673458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:01.673459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:01.673467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:01.674802 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.675331 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.675333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.675333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.675392 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.675684 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.675686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.675686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.675811 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:01.675873 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.676191 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.676288 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:52:01.676560 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.667123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45323 (local address 127.9.215.2:39029)
0811 18:52:01.667335 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:01.667381 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:01.667992 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:01.668275 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:01.668279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:01.668895 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:01.669114 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.669119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.669857 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.669858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.670720 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.670724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.671128 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.671134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.671268 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.671844 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:01.671854 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:01.673354 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:01.674908 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:01.674919 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:01.674922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:01.675233 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:01.675482 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:01.675483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:01.675483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:01.675620 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:01.675874 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:01.675883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:01.676043 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 18:52:01.676131 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:52:01.676321 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260811 18:52:01.677074 14171 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45323
I20260811 18:52:01.677191 14171 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:01.677392 14171 heartbeater.cc:511] Master 127.9.215.62:45323 requested a full tablet report, sending...
I20260811 18:52:01.677776 13542 ts_manager.cc:194] Registered new tserver with Master: 3c47ace89a7641359fefa16f0c3bc661 (127.9.215.2:42889)
I20260811 18:52:01.678164 13542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:39029
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:52:01.703811 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45323
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:01.790570 14177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:01.790815 14177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:01.790843 14177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:01.792114 14177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:01.792165 14177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:01.792194 14177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:52:01.793797 14177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:01.794052 14177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:01.794305 14177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:01.799844 14187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:01.799844 14192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:01.800247 14177 server_base.cc:1034] running on GCE node
W20260811 18:52:01.799866 14189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:01.800563 14177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:01.800880 14177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:01.802034 14177 hybrid_clock.cc:648] HybridClock initialized: now 1786474321802021 us; error 34 us; skew 500 ppm
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:01.804780 14177 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:52:01.805290 14177 webserver.cc:460] Webserver started at http://127.9.215.3:38637/ using document root <none> and password file <none>
I20260811 18:52:01.805505 14177 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:01.805557 14177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:01.805689 14177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:01.806810 14177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "5df1f9bf0e634414bc1034036e3a2c51"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "7f2a578858a7d4babd70be02a154513a"
server_key_iv: "c69ff1cd370a4c8c85d3acb7d623e05e"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.807163 14177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "5df1f9bf0e634414bc1034036e3a2c51"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "7f2a578858a7d4babd70be02a154513a"
server_key_iv: "c69ff1cd370a4c8c85d3acb7d623e05e"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.808712 14177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:01.809566 14207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.809767 14177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:01.809862 14177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "5df1f9bf0e634414bc1034036e3a2c51"
format_stamp: "Formatted at 2026-08-11 18:52:01 on dist-test-slave-pww0"
server_key: "7f2a578858a7d4babd70be02a154513a"
server_key_iv: "c69ff1cd370a4c8c85d3acb7d623e05e"
server_key_version: "encryptionkey@0"
I20260811 18:52:01.809930 14177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:01.826028 14177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:01.828640 14177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:01.828816 14177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:01.829085 14177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:01.829420 14177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:01.829457 14177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.829489 14177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:01.829505 14177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:01.835754 14177 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:40547
I20260811 18:52:01.835812 14436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:40547 every 8 connection(s)
I20260811 18:52:01.836160 14177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:01.840435 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 14177
I20260811 18:52:01.840546 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:52:01.840703 10076 external_mini_cluster.cc:1428] Setting key 55007da2728dfe90975a94288b7e7b10
Aug 11 18:52:01 dist-test-slave-pww0 krb5kdc[13465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474321, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:01.846292 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.837011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:38781 (local address 127.9.215.62:45323)
0811 18:52:01.837118 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.837121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.837320 (+ 199us) server_negotiation.cc:407] Connection header received
0811 18:52:01.837858 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.837860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.837873 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.837923 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:01.838965 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.839525 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.840763 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.841050 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.841125 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.843548 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:01.843550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:01.843550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:01.843558 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:01.844887 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.845311 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.845312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.845313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.845369 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.845623 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.845624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.845624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.845747 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:01.845807 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.846104 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.846197 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:52:01.846481 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.836946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45323 (local address 127.9.215.3:38781)
0811 18:52:01.837184 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:01.837227 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:01.837750 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:01.838009 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:01.838014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:01.838626 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:01.838852 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.838858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.839622 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.839623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.840671 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.840675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.841143 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.841144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.841315 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.841941 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:01.841950 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:01.843415 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:01.844975 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:01.844985 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:01.844987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:01.845235 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:01.845426 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:01.845427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:01.845428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:01.845551 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:01.845807 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:01.845815 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:01.845969 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 18:52:01.846062 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:52:01.846243 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260811 18:52:01.847164 14437 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45323
I20260811 18:52:01.847271 14437 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:01.847534 14437 heartbeater.cc:511] Master 127.9.215.62:45323 requested a full tablet report, sending...
I20260811 18:52:01.847999 13542 ts_manager.cc:194] Registered new tserver with Master: 5df1f9bf0e634414bc1034036e3a2c51 (127.9.215.3:40547)
I20260811 18:52:01.848385 13542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:38781
I20260811 18:52:01.852960 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:01.862053 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.854831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.9.215.62:45323)
0811 18:52:01.854881 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.854883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.855005 (+ 122us) server_negotiation.cc:407] Connection header received
0811 18:52:01.855056 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.855057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.855069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.855130 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:01.855793 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.856373 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.857349 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.857623 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.857688 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.859243 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:01.859245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:01.859245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:01.859254 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:01.860697 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.861184 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.861185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.861186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.861249 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:01.861439 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:01.861439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:01.861440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:01.861630 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:01.861722 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.861919 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.861980 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:52:01.864575 13542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:01.866406 13542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:01.873812 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.869309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43877 (local address 127.0.0.1:34508)
0811 18:52:01.869840 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:01.869847 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:01.869881 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:01.870090 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:01.870093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:01.870095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:01.870182 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.870185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.871606 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.871607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.872569 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.872573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.873483 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.873484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.873569 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.873590 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:52:01.873659 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:52:01.873719 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":123,"thread_start_us":89,"threads_started":1}
I20260811 18:52:01.873908 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.869590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.9.215.3:40547)
0811 18:52:01.869973 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.869977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.869986 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:01.870017 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.870018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.870030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.870105 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:01.870472 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.871276 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.872987 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.873533 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.873615 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.873676 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.873788 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.873828 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":121,"threads_started":1}
I20260811 18:52:01.873957 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.869314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:40547 (local address 127.0.0.1:59404)
0811 18:52:01.869733 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:01.869742 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:01.869784 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:01.870115 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:01.870116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:01.870118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:01.870185 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.870187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.871479 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.871481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.872896 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.872899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.873628 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.873629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.873695 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.873700 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:52:01.873759 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:52:01.873811 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":37,"thread_start_us":84,"threads_started":1}
I20260811 18:52:01.874202 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.869592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34508 (local address 127.9.215.1:43877)
0811 18:52:01.869763 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.869768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.869898 (+ 130us) server_negotiation.cc:407] Connection header received
0811 18:52:01.869947 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.869950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.869967 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.870052 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:01.870292 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.871487 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.872657 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.873390 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.873464 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.873539 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.874065 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.874125 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260811 18:52:01.874212 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.869309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42889 (local address 127.0.0.1:32878)
0811 18:52:01.869511 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:01.869556 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:01.869629 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:01.870392 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:01.870393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:01.870395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:01.870433 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.870434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.871728 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.871729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.872944 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:01.872947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.873827 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.873827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:01.873884 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.873887 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:01.873916 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 18:52:01.873953 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":26,"thread_start_us":95,"threads_started":1}
I20260811 18:52:01.874502 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:01.869797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.9.215.2:42889)
0811 18:52:01.870204 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:01.870208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:01.870217 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:01.870255 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:01.870275 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:01.870292 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:01.870499 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:01.870584 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.871666 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.873042 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:01.873741 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:01.874214 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:01.874378 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:01.874402 (+ 24us) server_negotiation.cc:299] Negotiation successful
0811 18:52:01.874441 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":79,"threads_started":1}
I20260811 18:52:01.875032 14298 tablet_service.cc:1467] Processing CreateTablet for tablet a363db496d494709bac642f374968e8e (DEFAULT_TABLE table=test-table [id=a0be8c51a7c64ff7803a532166844ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:01.874933 13785 tablet_service.cc:1467] Processing CreateTablet for tablet a363db496d494709bac642f374968e8e (DEFAULT_TABLE table=test-table [id=a0be8c51a7c64ff7803a532166844ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:01.875088 14037 tablet_service.cc:1467] Processing CreateTablet for tablet a363db496d494709bac642f374968e8e (DEFAULT_TABLE table=test-table [id=a0be8c51a7c64ff7803a532166844ec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:01.875447 13785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a363db496d494709bac642f374968e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:01.875480 14037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a363db496d494709bac642f374968e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:01.875499 14298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a363db496d494709bac642f374968e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:01.878615 14468 tablet_bootstrap.cc:492] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Bootstrap starting.
I20260811 18:52:01.878881 14469 tablet_bootstrap.cc:492] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: Bootstrap starting.
I20260811 18:52:01.879500 14468 tablet_bootstrap.cc:654] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:01.879523 14469 tablet_bootstrap.cc:654] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:01.880136 14468 log.cc:824] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:01.880136 14469 log.cc:824] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:01.880541 14472 tablet_bootstrap.cc:492] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Bootstrap starting.
I20260811 18:52:01.881217 14472 tablet_bootstrap.cc:654] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:01.881278 14468 tablet_bootstrap.cc:492] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: No bootstrap required, opened a new log
I20260811 18:52:01.881351 14468 ts_tablet_manager.cc:1397] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 18:52:01.881804 14472 log.cc:824] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:01.882409 14469 tablet_bootstrap.cc:492] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: No bootstrap required, opened a new log
I20260811 18:52:01.882493 14469 ts_tablet_manager.cc:1397] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 18:52:01.882743 14472 tablet_bootstrap.cc:492] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: No bootstrap required, opened a new log
I20260811 18:52:01.882814 14472 ts_tablet_manager.cc:1397] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:01.883502 14468 raft_consensus.cc:348] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:01.883724 14468 raft_consensus.cc:374] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:01.883773 14468 raft_consensus.cc:729] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c47ace89a7641359fefa16f0c3bc661, State: Initialized, Role: FOLLOWER
I20260811 18:52:01.883931 14468 consensus_queue.cc:260] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:01.884274 14468 ts_tablet_manager.cc:1428] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:52:01.884295 14171 heartbeater.cc:503] Master 127.9.215.62:45323 was elected leader, sending a full tablet report...
I20260811 18:52:01.884567 14469 raft_consensus.cc:348] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:01.884603 14472 raft_consensus.cc:348] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:01.884756 14469 raft_consensus.cc:374] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:01.884784 14469 raft_consensus.cc:729] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f3a0a582ec4968b0de67544a26b795, State: Initialized, Role: FOLLOWER
I20260811 18:52:01.884776 14472 raft_consensus.cc:374] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:01.884812 14472 raft_consensus.cc:729] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df1f9bf0e634414bc1034036e3a2c51, State: Initialized, Role: FOLLOWER
I20260811 18:52:01.884882 14469 consensus_queue.cc:260] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:01.884963 14472 consensus_queue.cc:260] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:01.885176 14469 ts_tablet_manager.cc:1428] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:52:01.885201 13919 heartbeater.cc:503] Master 127.9.215.62:45323 was elected leader, sending a full tablet report...
I20260811 18:52:01.885262 14472 ts_tablet_manager.cc:1428] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:52:01.885380 14437 heartbeater.cc:503] Master 127.9.215.62:45323 was elected leader, sending a full tablet report...
W20260811 18:52:01.916996 14172 tablet.cc:2369] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:01.981912 13920 tablet.cc:2369] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:02.086790 14439 tablet.cc:2369] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:02.171046 14482 raft_consensus.cc:482] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:02.171211 14482 raft_consensus.cc:504] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:02.171640 14482 leader_election.cc:288] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5df1f9bf0e634414bc1034036e3a2c51 (127.9.215.3:40547), 3c47ace89a7641359fefa16f0c3bc661 (127.9.215.2:42889)
I20260811 18:52:02.174827 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.171909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:42889 (local address 127.9.215.1:45379)
0811 18:52:02.172065 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.172077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.172146 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.172283 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.172284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.172286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:02.172362 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.172364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.173019 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.173020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.173926 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.173930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.174540 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.174541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.174610 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.174622 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.174671 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.174724 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260811 18:52:02.174813 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.172008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:45379 (local address 127.9.215.2:42889)
0811 18:52:02.172070 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.172073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.172163 (+ 90us) server_negotiation.cc:407] Connection header received
0811 18:52:02.172200 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.172201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.172215 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.172263 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:02.172436 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.172920 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.174046 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.174445 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.174514 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.174529 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.174697 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.174742 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:52:02.175099 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.172024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:46953 (local address 127.9.215.3:40547)
0811 18:52:02.172073 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.172074 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.172176 (+ 102us) server_negotiation.cc:407] Connection header received
0811 18:52:02.172234 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.172235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.172247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.172299 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:02.172470 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.172938 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.174048 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.174478 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.174633 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.174650 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.174907 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.174971 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:02.175135 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.171909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:40547 (local address 127.9.215.1:46953)
0811 18:52:02.172079 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.172090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.172174 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.172303 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.172303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.172305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:02.172399 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.172402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.173030 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.173031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.173935 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.173938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.174634 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.174636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.174719 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.174724 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.174850 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.175055 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":54,"thread_start_us":65,"threads_started":1}
I20260811 18:52:02.175220 14074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363db496d494709bac642f374968e8e" candidate_uuid: "53f3a0a582ec4968b0de67544a26b795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c47ace89a7641359fefa16f0c3bc661" is_pre_election: true
I20260811 18:52:02.175411 14074 raft_consensus.cc:2454] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f3a0a582ec4968b0de67544a26b795 in term 0.
I20260811 18:52:02.175648 14332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363db496d494709bac642f374968e8e" candidate_uuid: "53f3a0a582ec4968b0de67544a26b795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df1f9bf0e634414bc1034036e3a2c51" is_pre_election: true
I20260811 18:52:02.175719 13704 leader_election.cc:302] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c47ace89a7641359fefa16f0c3bc661, 53f3a0a582ec4968b0de67544a26b795; no voters:
I20260811 18:52:02.175779 14332 raft_consensus.cc:2454] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f3a0a582ec4968b0de67544a26b795 in term 0.
I20260811 18:52:02.175860 14482 raft_consensus.cc:2781] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:02.175928 14482 raft_consensus.cc:482] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:02.175952 14482 raft_consensus.cc:3037] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:02.176818 14482 raft_consensus.cc:504] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:02.176962 14482 leader_election.cc:288] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [CANDIDATE]: Term 1 election: Requested vote from peers 5df1f9bf0e634414bc1034036e3a2c51 (127.9.215.3:40547), 3c47ace89a7641359fefa16f0c3bc661 (127.9.215.2:42889)
I20260811 18:52:02.177285 14332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363db496d494709bac642f374968e8e" candidate_uuid: "53f3a0a582ec4968b0de67544a26b795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df1f9bf0e634414bc1034036e3a2c51"
I20260811 18:52:02.177335 14074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a363db496d494709bac642f374968e8e" candidate_uuid: "53f3a0a582ec4968b0de67544a26b795" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c47ace89a7641359fefa16f0c3bc661"
I20260811 18:52:02.177414 14074 raft_consensus.cc:3037] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:02.177407 14332 raft_consensus.cc:3037] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:02.178387 14332 raft_consensus.cc:2454] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f3a0a582ec4968b0de67544a26b795 in term 1.
I20260811 18:52:02.178388 14074 raft_consensus.cc:2454] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f3a0a582ec4968b0de67544a26b795 in term 1.
I20260811 18:52:02.178604 13704 leader_election.cc:302] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c47ace89a7641359fefa16f0c3bc661, 53f3a0a582ec4968b0de67544a26b795; no voters:
I20260811 18:52:02.178710 14482 raft_consensus.cc:2781] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:02.178911 14482 raft_consensus.cc:686] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [term 1 LEADER]: Becoming Leader. State: Replica: 53f3a0a582ec4968b0de67544a26b795, State: Running, Role: LEADER
I20260811 18:52:02.179024 14482 consensus_queue.cc:237] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } }
I20260811 18:52:02.179952 13542 catalog_manager.cc:5495] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53f3a0a582ec4968b0de67544a26b795 (127.9.215.1). New cstate: current_term: 1 leader_uuid: "53f3a0a582ec4968b0de67544a26b795" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f3a0a582ec4968b0de67544a26b795" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 } health_report { overall_health: UNKNOWN } } }
I20260811 18:52:02.229949 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.227523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34522 (local address 127.9.215.1:43877)
0811 18:52:02.227568 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.227569 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.227598 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:52:02.227682 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.227682 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.227703 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.227778 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:02.228023 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.228512 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.229285 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.229480 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.229532 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.229659 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:02.229749 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:02.229817 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.229874 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.229884 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:02.232039 14332 raft_consensus.cc:1261] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 1 FOLLOWER]: Refusing update from remote peer 53f3a0a582ec4968b0de67544a26b795: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:02.232041 14074 raft_consensus.cc:1261] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 1 FOLLOWER]: Refusing update from remote peer 53f3a0a582ec4968b0de67544a26b795: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:02.232491 14490 consensus_queue.cc:1035] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df1f9bf0e634414bc1034036e3a2c51" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 40547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:02.232643 14482 consensus_queue.cc:1035] T a363db496d494709bac642f374968e8e P 53f3a0a582ec4968b0de67544a26b795 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c47ace89a7641359fefa16f0c3bc661" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 42889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:02.235127 14495 mvcc.cc:205] Tried to move back new op lower bound from 7317398823859281920 to 7317398823645954048. Current Snapshot: MvccSnapshot[applied={T|T < 7317398823859281920}]
I20260811 18:52:02.235759 14500 mvcc.cc:205] Tried to move back new op lower bound from 7317398823859281920 to 7317398823645954048. Current Snapshot: MvccSnapshot[applied={T|T < 7317398823859281920}]
I20260811 18:52:02.236423 14496 mvcc.cc:205] Tried to move back new op lower bound from 7317398823859281920 to 7317398823645954048. Current Snapshot: MvccSnapshot[applied={T|T < 7317398823859281920}]
I20260811 18:52:02.237793 13540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:02.237905 13540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:02.238746 13540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e344e6cdd13b4be99cc747481643daf3: Sending DeleteTablet for 3 replicas of tablet a363db496d494709bac642f374968e8e
I20260811 18:52:02.239104 14298 tablet_service.cc:1514] Processing DeleteTablet for tablet a363db496d494709bac642f374968e8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:02 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:59404
I20260811 18:52:02.239153 14037 tablet_service.cc:1514] Processing DeleteTablet for tablet a363db496d494709bac642f374968e8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:02 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:32878
I20260811 18:52:02.239437 14519 tablet_replica.cc:330] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: stopping tablet replica
I20260811 18:52:02.239578 14519 raft_consensus.cc:2229] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:02.239557 13785 tablet_service.cc:1514] Processing DeleteTablet for tablet a363db496d494709bac642f374968e8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:02 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:34508
I20260811 18:52:02.239589 14513 tablet_replica.cc:330] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: stopping tablet replica
I20260811 18:52:02.239714 14519 raft_consensus.cc:2258] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:02.239816 14513 raft_consensus.cc:2229] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:02.239818 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13669
I20260811 18:52:02.243297 14513 raft_consensus.cc:2258] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:02.244248 14519 ts_tablet_manager.cc:1905] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:02.246668 14519 ts_tablet_manager.cc:1918] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:02.246781 14519 log.cc:1196] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/a363db496d494709bac642f374968e8e
I20260811 18:52:02.247108 14519 ts_tablet_manager.cc:1939] T a363db496d494709bac642f374968e8e P 3c47ace89a7641359fefa16f0c3bc661: Deleting consensus metadata
I20260811 18:52:02.251869 14513 ts_tablet_manager.cc:1905] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:02.253700 14513 ts_tablet_manager.cc:1918] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:02.253821 14513 log.cc:1196] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/a363db496d494709bac642f374968e8e
I20260811 18:52:02.254408 14513 ts_tablet_manager.cc:1939] T a363db496d494709bac642f374968e8e P 5df1f9bf0e634414bc1034036e3a2c51: Deleting consensus metadata
I20260811 18:52:02.255023 13518 catalog_manager.cc:4841] TS 5df1f9bf0e634414bc1034036e3a2c51 (127.9.215.3:40547): tablet a363db496d494709bac642f374968e8e (table test-table [id=a0be8c51a7c64ff7803a532166844ec3]) successfully deleted
I20260811 18:52:02.258298 13523 catalog_manager.cc:4841] TS 3c47ace89a7641359fefa16f0c3bc661 (127.9.215.2:42889): tablet a363db496d494709bac642f374968e8e (table test-table [id=a0be8c51a7c64ff7803a532166844ec3]) successfully deleted
W20260811 18:52:02.262449 13520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:43877 (error 108)
I20260811 18:52:02.262611 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13925
I20260811 18:52:02.283526 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.283287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43877 (local address 127.0.0.1:34524)
0811 18:52:02.283371 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.283447 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 18:52:02.283763 13520 catalog_manager.cc:4584] TS 53f3a0a582ec4968b0de67544a26b795 (127.9.215.1:43877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a363db496d494709bac642f374968e8e: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43877: connect: Connection refused (error 111)
I20260811 18:52:02.296610 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 14177
I20260811 18:52:02.320901 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 13495
2026-08-11T18:52:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2591 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T18:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:02Z Disabled control of system clock
I20260811 18:52:02.373245 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45203
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:33625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:45203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:02.457732 14527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:02.457995 14527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:02.458019 14527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:02.459334 14527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:02.459390 14527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:02.459401 14527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:02.459414 14527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:02.459425 14527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:02.461362 14527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:45203
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:45203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:02.461686 14527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:02.462047 14527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:02.467115 14536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:02.467146 14541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.467348 14527 server_base.cc:1034] running on GCE node
W20260811 18:52:02.467168 14535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.467717 14527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:02.468032 14527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:02.469218 14527 hybrid_clock.cc:648] HybridClock initialized: now 1786474322469187 us; error 39 us; skew 500 ppm
I20260811 18:52:02.470567 14527 webserver.cc:460] Webserver started at http://127.9.215.62:44181/ using document root <none> and password file <none>
I20260811 18:52:02.470825 14527 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:02.470880 14527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:02.471011 14527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:02.472193 14527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "3b11ccc46712427ea503983531ed3539"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "c6b54cfd1c9e1fa78b44159ec7e5f6ac"
server_key_iv: "ce555d89b72a2bc69d6bb17a7d4ed1af"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.472543 14527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "3b11ccc46712427ea503983531ed3539"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "c6b54cfd1c9e1fa78b44159ec7e5f6ac"
server_key_iv: "ce555d89b72a2bc69d6bb17a7d4ed1af"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.473910 14527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:02.474721 14550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.474937 14527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:02.475044 14527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "3b11ccc46712427ea503983531ed3539"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "c6b54cfd1c9e1fa78b44159ec7e5f6ac"
server_key_iv: "ce555d89b72a2bc69d6bb17a7d4ed1af"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.475113 14527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:02.491833 14527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:02.492195 14527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:02.492354 14527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:02.496592 14527 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:45203
I20260811 18:52:02.496675 14652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:45203 every 8 connection(s)
I20260811 18:52:02.497037 14527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:02.498193 14653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:02.499231 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 14527
I20260811 18:52:02.499338 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:02.499607 10076 external_mini_cluster.cc:1428] Setting key ec9f66d736b4358da16e3fb4edcfdc86
I20260811 18:52:02.501133 14653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539: Bootstrap starting.
I20260811 18:52:02.501780 14653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:02.502068 14653 log.cc:824] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:02.502882 14653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539: No bootstrap required, opened a new log
I20260811 18:52:02.505100 14653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b11ccc46712427ea503983531ed3539" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45203 } }
I20260811 18:52:02.505278 14653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:02.505314 14653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b11ccc46712427ea503983531ed3539, State: Initialized, Role: FOLLOWER
I20260811 18:52:02.505455 14653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b11ccc46712427ea503983531ed3539" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45203 } }
I20260811 18:52:02.505542 14653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:02.505597 14653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:02.505650 14653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:02.505826 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.500196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.9.215.62:45203)
0811 18:52:02.500430 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.500438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.500466 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:52:02.500945 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.500953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.501020 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.501155 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:02.501433 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.502510 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.503378 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.503898 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.504021 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.504940 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:02.504945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:02.504945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:02.505123 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:02.505160 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:02.505172 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:02.505237 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.505335 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.505526 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260811 18:52:02.506458 14653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b11ccc46712427ea503983531ed3539" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45203 } }
I20260811 18:52:02.506573 14653 leader_election.cc:302] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b11ccc46712427ea503983531ed3539; no voters:
I20260811 18:52:02.506776 14653 leader_election.cc:288] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:02.506827 14665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:02.506995 14665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [term 1 LEADER]: Becoming Leader. State: Replica: 3b11ccc46712427ea503983531ed3539, State: Running, Role: LEADER
I20260811 18:52:02.507054 14653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:02.507091 14665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b11ccc46712427ea503983531ed3539" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45203 } }
I20260811 18:52:02.507478 14666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b11ccc46712427ea503983531ed3539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b11ccc46712427ea503983531ed3539" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45203 } } }
I20260811 18:52:02.507490 14669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b11ccc46712427ea503983531ed3539. Latest consensus state: current_term: 1 leader_uuid: "3b11ccc46712427ea503983531ed3539" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b11ccc46712427ea503983531ed3539" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 45203 } } }
I20260811 18:52:02.507582 14666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:02.507588 14669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:02.507854 14679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:02.508334 14679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:02.509980 14679 catalog_manager.cc:1347] Generated new cluster ID: 41009e314e95433784e12bc10c059412
I20260811 18:52:02.510027 14679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:02.528018 14679 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:02.528954 14679 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:02.535156 14679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539: Generated new TSK 0
I20260811 18:52:02.535486 14679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:02.537585 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45203
--builtin_ntp_servers=127.9.215.20:33625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:02.624920 14701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:02.625155 14701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:02.625190 14701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:02.626435 14701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:02.626485 14701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:02.626523 14701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:52:02.628173 14701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:02.628459 14701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:02.628786 14701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:02.633975 14723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.634153 14701 server_base.cc:1034] running on GCE node
W20260811 18:52:02.634011 14715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:02.633997 14714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.634606 14701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:02.634866 14701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:02.636055 14701 hybrid_clock.cc:648] HybridClock initialized: now 1786474322636034 us; error 33 us; skew 500 ppm
I20260811 18:52:02.637485 14701 webserver.cc:460] Webserver started at http://127.9.215.1:34473/ using document root <none> and password file <none>
I20260811 18:52:02.637768 14701 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:02.637820 14701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:02.637951 14701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:02.639042 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "eb0a3bd4f9bf5a4037958cacdda1f20f"
server_key_iv: "368824d70bf9a93e3e2d6cf480f93ffc"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.639393 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "eb0a3bd4f9bf5a4037958cacdda1f20f"
server_key_iv: "368824d70bf9a93e3e2d6cf480f93ffc"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.640780 14701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:02.641691 14733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.641907 14701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:02.642006 14701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "eb0a3bd4f9bf5a4037958cacdda1f20f"
server_key_iv: "368824d70bf9a93e3e2d6cf480f93ffc"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.642071 14701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:02.655333 14701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:02.655738 14701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:02.655902 14701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:02.656144 14701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:02.656689 14701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:02.656740 14701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.656766 14701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:02.656790 14701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.662740 14701 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:42931
I20260811 18:52:02.662818 14949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:42931 every 8 connection(s)
I20260811 18:52:02.663173 14701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:02.663810 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 14701
I20260811 18:52:02.663920 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:52:02.664157 10076 external_mini_cluster.cc:1428] Setting key c12011fed395706a1dbfa686f78bd825
I20260811 18:52:02.665275 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45203
--builtin_ntp_servers=127.9.215.20:33625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:52:02.669812 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.664453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:40385 (local address 127.9.215.62:45203)
0811 18:52:02.664498 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.664500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.664617 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:52:02.665263 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.665264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.665276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.665329 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:02.666139 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.666641 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.668138 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.668347 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.668412 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.669472 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:02.669473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:02.669474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:02.669479 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:02.669497 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:02.669501 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:02.669539 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.669698 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.669729 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:52:02.670053 14952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.664111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45203 (local address 127.9.215.1:40385)
0811 18:52:02.664442 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.664493 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.665149 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.665423 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.665427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.665439 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:02.666044 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.666050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.666742 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.666743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.668019 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.668023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.668451 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.668452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.668634 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.669280 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:52:02.669281 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:02.669324 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:52:02.669358 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:02.669573 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:02.669582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:02.669585 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.669665 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.669799 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":108,"threads_started":1}
I20260811 18:52:02.670708 14950 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45203
I20260811 18:52:02.670811 14950 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:02.671007 14950 heartbeater.cc:511] Master 127.9.215.62:45203 requested a full tablet report, sending...
I20260811 18:52:02.671659 14579 ts_manager.cc:194] Registered new tserver with Master: cca4e6dcafbc4cfa81aa9dae0678b1fd (127.9.215.1:42931)
I20260811 18:52:02.672622 14579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.1:40385
W20260811 18:52:02.753197 14956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:02.753433 14956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:02.753458 14956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:02.754652 14956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:02.754688 14956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:02.754714 14956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:52:02.756107 14956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:02.756381 14956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:02.756804 14956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:02.762338 14964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:02.762338 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.762466 14956 server_base.cc:1034] running on GCE node
W20260811 18:52:02.762341 14967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.762971 14956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:02.763295 14956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:02.764451 14956 hybrid_clock.cc:648] HybridClock initialized: now 1786474322764425 us; error 41 us; skew 500 ppm
I20260811 18:52:02.765836 14956 webserver.cc:460] Webserver started at http://127.9.215.2:41585/ using document root <none> and password file <none>
I20260811 18:52:02.766085 14956 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:02.766141 14956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:02.766276 14956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:02.767402 14956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "9a9a2377ec824fef3d8f4cec3813ca98"
server_key_iv: "dff0340cd0150f0c3d5d1c65756d4b5b"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.767835 14956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "9a9a2377ec824fef3d8f4cec3813ca98"
server_key_iv: "dff0340cd0150f0c3d5d1c65756d4b5b"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.769284 14956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:02.770112 14977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.770303 14956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:02.770406 14956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "9a9a2377ec824fef3d8f4cec3813ca98"
server_key_iv: "dff0340cd0150f0c3d5d1c65756d4b5b"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.770483 14956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:02.795881 14956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:02.796279 14956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:02.796432 14956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:02.796691 14956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:02.797047 14956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:02.797093 14956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.797125 14956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:02.797142 14956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.803061 14956 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:38249
I20260811 18:52:02.803131 15210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:38249 every 8 connection(s)
I20260811 18:52:02.803501 14956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:02.809115 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.804320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:37779 (local address 127.9.215.62:45203)
0811 18:52:02.804387 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.804389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.804627 (+ 238us) server_negotiation.cc:407] Connection header received
0811 18:52:02.805193 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.805194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.805206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.805268 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:02.806002 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.806607 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.807807 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.808009 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.808072 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.808886 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:02.808887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:02.808887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:02.808893 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:02.808909 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:02.808913 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:02.808954 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.809021 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.809047 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:52:02.809343 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.804313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45203 (local address 127.9.215.2:37779)
0811 18:52:02.804499 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.804537 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.805060 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.805341 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.805345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.805354 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:02.805898 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.805904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.806736 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.806737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.807730 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.807734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.808080 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.808081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.808205 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.808760 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:52:02.808761 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:02.808795 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:52:02.808823 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:02.808960 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:02.808963 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:02.808966 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.809014 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.809104 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260811 18:52:02.809988 15214 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45203
I20260811 18:52:02.810091 15214 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:02.810287 15214 heartbeater.cc:511] Master 127.9.215.62:45203 requested a full tablet report, sending...
I20260811 18:52:02.810707 14579 ts_manager.cc:194] Registered new tserver with Master: 308d7a73bf7e4b039f9cd15b2d5e98f0 (127.9.215.2:38249)
I20260811 18:52:02.811058 14579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.2:37779
I20260811 18:52:02.812294 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 14956
I20260811 18:52:02.812382 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:52:02.812546 10076 external_mini_cluster.cc:1428] Setting key b0b0095dc6a865c517a566c61239e0b2
I20260811 18:52:02.813753 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45203
--builtin_ntp_servers=127.9.215.20:33625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:02.900590 15219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:02.900820 15219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:02.900843 15219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:02.902055 15219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:02.902096 15219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:02.902123 15219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:52:02.903575 15219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:33625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:45203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:02.903817 15219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:02.904152 15219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:02.909577 15232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:02.909577 15228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:02.909577 15229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:02.909910 15219 server_base.cc:1034] running on GCE node
I20260811 18:52:02.910142 15219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:02.910454 15219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:02.911607 15219 hybrid_clock.cc:648] HybridClock initialized: now 1786474322911567 us; error 51 us; skew 500 ppm
I20260811 18:52:02.913091 15219 webserver.cc:460] Webserver started at http://127.9.215.3:33273/ using document root <none> and password file <none>
I20260811 18:52:02.913358 15219 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:02.913409 15219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:02.913532 15219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:02.914702 15219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "050d0ea9b4294285884a842f653286de"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "0bdbf2bd1bbacd958ce9c2427bd25945"
server_key_iv: "1593acb40fdd7327af85742222dd30a3"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.915057 15219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "050d0ea9b4294285884a842f653286de"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "0bdbf2bd1bbacd958ce9c2427bd25945"
server_key_iv: "1593acb40fdd7327af85742222dd30a3"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.916477 15219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:02.917315 15240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.917488 15219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:02.917582 15219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "050d0ea9b4294285884a842f653286de"
format_stamp: "Formatted at 2026-08-11 18:52:02 on dist-test-slave-pww0"
server_key: "0bdbf2bd1bbacd958ce9c2427bd25945"
server_key_iv: "1593acb40fdd7327af85742222dd30a3"
server_key_version: "encryptionkey@0"
I20260811 18:52:02.917650 15219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:02.945148 15219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:02.945500 15219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:02.945675 15219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:02.945932 15219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:02.946280 15219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:02.946314 15219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.946348 15219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:02.946365 15219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:02.954298 15219 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:42829
I20260811 18:52:02.954376 15465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:42829 every 8 connection(s)
I20260811 18:52:02.954762 15219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:02.960712 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15219
I20260811 18:52:02.960841 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:52:02.961024 10076 external_mini_cluster.cc:1428] Setting key 21f1d8973190e7bfa6c3e86851f8736f
I20260811 18:52:02.961179 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.955663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:57803 (local address 127.9.215.62:45203)
0811 18:52:02.955711 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.955713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.956113 (+ 400us) server_negotiation.cc:407] Connection header received
0811 18:52:02.956674 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.956675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.956691 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.956755 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:02.957583 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.958088 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.959283 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.959669 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.959766 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.960834 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:02.960836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:02.960836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:02.960845 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:02.960868 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:02.960874 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:02.960940 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.961058 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.961093 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:02.961472 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.955656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:45203 (local address 127.9.215.3:57803)
0811 18:52:02.955954 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.955995 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.956556 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.956863 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.956867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.956877 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:02.957473 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.957479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.958188 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.958189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.959162 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.959166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.959798 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.959800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.959957 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.960600 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:52:02.960601 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:02.960649 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:52:02.960688 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:02.960955 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:02.960966 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:02.960969 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.961042 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.961212 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260811 18:52:02.962059 15466 heartbeater.cc:347] Connected to a master server at 127.9.215.62:45203
I20260811 18:52:02.962224 15466 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:02.962410 15466 heartbeater.cc:511] Master 127.9.215.62:45203 requested a full tablet report, sending...
I20260811 18:52:02.962847 14579 ts_manager.cc:194] Registered new tserver with Master: 050d0ea9b4294285884a842f653286de (127.9.215.3:42829)
I20260811 18:52:02.963652 14579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.3:57803
I20260811 18:52:02.973735 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:02.979705 14662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.975692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.9.215.62:45203)
0811 18:52:02.975748 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.975750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.975887 (+ 137us) server_negotiation.cc:407] Connection header received
0811 18:52:02.975960 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.975961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.975973 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.976015 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:02.976235 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.976765 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.977942 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.978161 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.978226 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.979299 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:02.979301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:02.979301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:02.979307 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:02.979324 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:02.979329 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:02.979452 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.979601 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.979616 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:52:02.982295 14579 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:02.984125 14579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:02.992012 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.987723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:38249 (local address 127.0.0.1:52942)
0811 18:52:02.988044 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.988055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.988102 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.988519 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.988520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.988523 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:02.988609 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.988611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.989528 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.989530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.990723 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.990726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.991673 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.991674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.991773 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.991791 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.991859 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.991922 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":82,"threads_started":1}
I20260811 18:52:02.992025 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.987971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.9.215.2:38249)
0811 18:52:02.988282 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.988285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.988297 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:52:02.988341 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.988343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.988359 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.988439 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:02.988699 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.989432 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.990829 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.991593 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.991661 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.991773 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.991886 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.991926 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":94,"threads_started":1}
I20260811 18:52:02.992545 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.987738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:42829 (local address 127.0.0.1:41316)
0811 18:52:02.988203 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.988212 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.988259 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.988549 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.988550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.988552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:02.988612 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.988615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.990011 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.990012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.991014 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.991016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.992090 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.992091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.992354 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.992360 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.992413 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.992458 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":10,"thread_start_us":74,"threads_started":1}
I20260811 18:52:02.992573 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.987971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.9.215.3:42829)
0811 18:52:02.988181 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.988197 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.988328 (+ 131us) server_negotiation.cc:407] Connection header received
0811 18:52:02.988379 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.988381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.988397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.988471 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:02.988707 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.989901 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.991102 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.991880 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.991958 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.992050 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.992440 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.992491 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":85,"threads_started":1}
I20260811 18:52:02.992898 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.987658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42931 (local address 127.0.0.1:40798)
0811 18:52:02.987899 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:02.987949 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:02.988041 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:02.988496 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:02.988499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:02.988501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:02.988612 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.988616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.990062 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.990063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.991174 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:02.991178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.992446 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.992446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:02.992500 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.992503 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:02.992547 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 18:52:02.992582 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260811 18:52:02.992918 15067 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0efe533b1943e58f0664971f85e3d8 (DEFAULT_TABLE table=test-table [id=a07be8cb50d341068546068247605f10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:02.993153 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:02.987743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.9.215.1:42931)
0811 18:52:02.987961 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:02.987966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:02.988186 (+ 220us) server_negotiation.cc:407] Connection header received
0811 18:52:02.988303 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:02.988306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:02.988324 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:02.988398 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:02.988707 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.989901 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.991280 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:02.992062 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:02.992310 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:02.992419 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:02.993023 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 18:52:02.993075 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":91,"threads_started":1}
I20260811 18:52:02.993381 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0efe533b1943e58f0664971f85e3d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:02.993975 14821 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0efe533b1943e58f0664971f85e3d8 (DEFAULT_TABLE table=test-table [id=a07be8cb50d341068546068247605f10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:02.994434 14821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0efe533b1943e58f0664971f85e3d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:02.996665 15492 tablet_bootstrap.cc:492] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Bootstrap starting.
I20260811 18:52:02.997462 15492 tablet_bootstrap.cc:654] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:02.997956 15492 log.cc:824] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:02.998498 15495 tablet_bootstrap.cc:492] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: Bootstrap starting.
I20260811 18:52:02.998963 15347 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0efe533b1943e58f0664971f85e3d8 (DEFAULT_TABLE table=test-table [id=a07be8cb50d341068546068247605f10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:02.999275 15495 tablet_bootstrap.cc:654] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:02.999413 15347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f0efe533b1943e58f0664971f85e3d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:02.999681 15495 log.cc:824] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:03.000209 15492 tablet_bootstrap.cc:492] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: No bootstrap required, opened a new log
I20260811 18:52:03.000298 15492 ts_tablet_manager.cc:1397] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:52:03.000468 15495 tablet_bootstrap.cc:492] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: No bootstrap required, opened a new log
I20260811 18:52:03.000536 15495 ts_tablet_manager.cc:1397] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:03.002543 15495 raft_consensus.cc:348] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.002758 15495 raft_consensus.cc:374] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:03.002799 15495 raft_consensus.cc:729] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cca4e6dcafbc4cfa81aa9dae0678b1fd, State: Initialized, Role: FOLLOWER
I20260811 18:52:03.002956 15495 consensus_queue.cc:260] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.003252 15495 ts_tablet_manager.cc:1428] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:03.003394 14950 heartbeater.cc:503] Master 127.9.215.62:45203 was elected leader, sending a full tablet report...
I20260811 18:52:03.003367 15492 raft_consensus.cc:348] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.003580 15492 raft_consensus.cc:374] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:03.004161 15492 raft_consensus.cc:729] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308d7a73bf7e4b039f9cd15b2d5e98f0, State: Initialized, Role: FOLLOWER
I20260811 18:52:03.004338 15492 consensus_queue.cc:260] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.004662 15492 ts_tablet_manager.cc:1428] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 18:52:03.004696 15214 heartbeater.cc:503] Master 127.9.215.62:45203 was elected leader, sending a full tablet report...
I20260811 18:52:03.004884 15502 tablet_bootstrap.cc:492] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Bootstrap starting.
I20260811 18:52:03.005743 15502 tablet_bootstrap.cc:654] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:03.006153 15502 log.cc:824] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:03.007192 15502 tablet_bootstrap.cc:492] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: No bootstrap required, opened a new log
I20260811 18:52:03.007372 15502 ts_tablet_manager.cc:1397] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:52:03.009555 15502 raft_consensus.cc:348] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.009732 15502 raft_consensus.cc:374] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:03.009769 15502 raft_consensus.cc:729] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 050d0ea9b4294285884a842f653286de, State: Initialized, Role: FOLLOWER
I20260811 18:52:03.009892 15502 consensus_queue.cc:260] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.010200 15502 ts_tablet_manager.cc:1428] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:03.010319 15466 heartbeater.cc:503] Master 127.9.215.62:45203 was elected leader, sending a full tablet report...
W20260811 18:52:03.054148 15216 tablet.cc:2369] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:03.163990 14951 tablet.cc:2369] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:03.165010 15504 raft_consensus.cc:482] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:03.165143 15504 raft_consensus.cc:504] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.165462 15504 leader_election.cc:288] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 050d0ea9b4294285884a842f653286de (127.9.215.3:42829), cca4e6dcafbc4cfa81aa9dae0678b1fd (127.9.215.1:42931)
I20260811 18:52:03.169610 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:03.165776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:34377 (local address 127.9.215.3:42829)
0811 18:52:03.165832 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:03.165833 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:03.165992 (+ 159us) server_negotiation.cc:407] Connection header received
0811 18:52:03.166028 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:03.166029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:03.166036 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:03.166077 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:03.166221 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.166918 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.168593 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.169189 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.169262 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:03.169274 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:03.169480 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 18:52:03.169534 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:52:03.169622 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:03.165676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42931 (local address 127.9.215.2:43511)
0811 18:52:03.165759 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:03.165770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:03.165816 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:03.165977 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:03.165977 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:03.165979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:03.166061 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:03.166063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.167016 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.167018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:03.168456 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:03.168459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.169291 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.169292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:03.169387 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:03.169394 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:52:03.169460 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:52:03.169510 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":155}
I20260811 18:52:03.169629 15509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:03.165730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:42829 (local address 127.9.215.2:34377)
0811 18:52:03.165916 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:03.165923 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:03.165977 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:03.166098 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:03.166099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:03.166101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:03.166156 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:03.166157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.167021 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.167021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:03.168456 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:03.168459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.169291 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.169292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:03.169387 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:03.169394 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:52:03.169460 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:52:03.169510 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":49,"thread_start_us":134,"threads_started":1}
I20260811 18:52:03.169610 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:03.165759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:43511 (local address 127.9.215.1:42931)
0811 18:52:03.165808 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:03.165810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:03.165821 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:52:03.165885 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:03.165886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:03.165898 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:03.165953 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:03.166154 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.166902 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.168599 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.169177 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.169262 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:03.169274 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:03.169481 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 18:52:03.169532 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:52:03.170105 15368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0efe533b1943e58f0664971f85e3d8" candidate_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "050d0ea9b4294285884a842f653286de" is_pre_election: true
I20260811 18:52:03.170195 14858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0efe533b1943e58f0664971f85e3d8" candidate_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" is_pre_election: true
I20260811 18:52:03.170303 15368 raft_consensus.cc:2454] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308d7a73bf7e4b039f9cd15b2d5e98f0 in term 0.
I20260811 18:52:03.170384 14858 raft_consensus.cc:2454] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308d7a73bf7e4b039f9cd15b2d5e98f0 in term 0.
I20260811 18:52:03.170562 14983 leader_election.cc:302] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 050d0ea9b4294285884a842f653286de, 308d7a73bf7e4b039f9cd15b2d5e98f0; no voters:
I20260811 18:52:03.170733 15504 raft_consensus.cc:2781] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:03.170800 15504 raft_consensus.cc:482] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:03.170826 15504 raft_consensus.cc:3037] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:03.171612 15504 raft_consensus.cc:504] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.171762 15504 leader_election.cc:288] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [CANDIDATE]: Term 1 election: Requested vote from peers 050d0ea9b4294285884a842f653286de (127.9.215.3:42829), cca4e6dcafbc4cfa81aa9dae0678b1fd (127.9.215.1:42931)
I20260811 18:52:03.171962 14858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0efe533b1943e58f0664971f85e3d8" candidate_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd"
I20260811 18:52:03.172047 14858 raft_consensus.cc:3037] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:03.172047 15368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0efe533b1943e58f0664971f85e3d8" candidate_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "050d0ea9b4294285884a842f653286de"
I20260811 18:52:03.172133 15368 raft_consensus.cc:3037] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:03.172842 14858 raft_consensus.cc:2454] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308d7a73bf7e4b039f9cd15b2d5e98f0 in term 1.
I20260811 18:52:03.172861 15368 raft_consensus.cc:2454] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308d7a73bf7e4b039f9cd15b2d5e98f0 in term 1.
I20260811 18:52:03.173038 14979 leader_election.cc:302] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308d7a73bf7e4b039f9cd15b2d5e98f0, cca4e6dcafbc4cfa81aa9dae0678b1fd; no voters:
I20260811 18:52:03.173237 15504 raft_consensus.cc:2781] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:03.173434 15504 raft_consensus.cc:686] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 1 LEADER]: Becoming Leader. State: Replica: 308d7a73bf7e4b039f9cd15b2d5e98f0, State: Running, Role: LEADER
I20260811 18:52:03.173556 15504 consensus_queue.cc:237] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } }
I20260811 18:52:03.174512 14579 catalog_manager.cc:5495] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 308d7a73bf7e4b039f9cd15b2d5e98f0 (127.9.215.2). New cstate: current_term: 1 leader_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "308d7a73bf7e4b039f9cd15b2d5e98f0" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 38249 } health_report { overall_health: HEALTHY } } }
W20260811 18:52:03.205389 15467 tablet.cc:2369] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:03.224733 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:03.222315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.9.215.2:38249)
0811 18:52:03.222387 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:03.222389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:03.222398 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:03.222442 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:03.222443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:03.222466 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:03.222535 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:03.222678 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.223155 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.223957 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:03.224152 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:03.224211 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:03.224374 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:03.224470 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:03.224560 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:03.224616 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 18:52:03.224627 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:52:03.226868 15368 raft_consensus.cc:1261] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 1 FOLLOWER]: Refusing update from remote peer 308d7a73bf7e4b039f9cd15b2d5e98f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:03.226881 14858 raft_consensus.cc:1261] T 2f0efe533b1943e58f0664971f85e3d8 P cca4e6dcafbc4cfa81aa9dae0678b1fd [term 1 FOLLOWER]: Refusing update from remote peer 308d7a73bf7e4b039f9cd15b2d5e98f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:03.227252 15514 consensus_queue.cc:1035] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cca4e6dcafbc4cfa81aa9dae0678b1fd" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 42931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:03.227440 15504 consensus_queue.cc:1035] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "050d0ea9b4294285884a842f653286de" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 42829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:03.230048 15517 mvcc.cc:205] Tried to move back new op lower bound from 7317398827933847552 to 7317398827719528448. Current Snapshot: MvccSnapshot[applied={T|T < 7317398827933847552}]
I20260811 18:52:03.230384 15522 mvcc.cc:205] Tried to move back new op lower bound from 7317398827933847552 to 7317398827719528448. Current Snapshot: MvccSnapshot[applied={T|T < 7317398827933847552}]
I20260811 18:52:03.233423 14579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:03.233529 14579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:03.234311 14579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b11ccc46712427ea503983531ed3539: Sending DeleteTablet for 3 replicas of tablet 2f0efe533b1943e58f0664971f85e3d8
I20260811 18:52:03.234701 15067 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0efe533b1943e58f0664971f85e3d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:03 UTC) from {username='slave'} at 127.0.0.1:52942
I20260811 18:52:03.234687 14821 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0efe533b1943e58f0664971f85e3d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:03 UTC) from {username='slave'} at 127.0.0.1:40798
I20260811 18:52:03.234719 15347 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f0efe533b1943e58f0664971f85e3d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:03 UTC) from {username='slave'} at 127.0.0.1:41316
I20260811 18:52:03.234969 15545 tablet_replica.cc:330] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: stopping tablet replica
I20260811 18:52:03.235087 15545 raft_consensus.cc:2229] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:03.235200 15545 raft_consensus.cc:2258] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:03.235359 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 14701
I20260811 18:52:03.235716 15545 ts_tablet_manager.cc:1905] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:03.236172 15546 tablet_replica.cc:330] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: stopping tablet replica
I20260811 18:52:03.237471 15545 ts_tablet_manager.cc:1918] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:03.237553 15545 log.cc:1196] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/2f0efe533b1943e58f0664971f85e3d8
I20260811 18:52:03.237763 15545 ts_tablet_manager.cc:1939] T 2f0efe533b1943e58f0664971f85e3d8 P 050d0ea9b4294285884a842f653286de: Deleting consensus metadata
I20260811 18:52:03.238169 14557 catalog_manager.cc:4841] TS 050d0ea9b4294285884a842f653286de (127.9.215.3:42829): tablet 2f0efe533b1943e58f0664971f85e3d8 (table test-table [id=a07be8cb50d341068546068247605f10]) successfully deleted
I20260811 18:52:03.239444 15546 raft_consensus.cc:2229] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:52:03.239675 15546 raft_consensus.cc:2258] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:03.242369 15546 ts_tablet_manager.cc:1905] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:03.244153 15546 ts_tablet_manager.cc:1918] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:03.244232 15546 log.cc:1196] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/2f0efe533b1943e58f0664971f85e3d8
I20260811 18:52:03.244452 15546 ts_tablet_manager.cc:1939] T 2f0efe533b1943e58f0664971f85e3d8 P 308d7a73bf7e4b039f9cd15b2d5e98f0: Deleting consensus metadata
I20260811 18:52:03.245061 14556 catalog_manager.cc:4841] TS 308d7a73bf7e4b039f9cd15b2d5e98f0 (127.9.215.2:38249): tablet 2f0efe533b1943e58f0664971f85e3d8 (table test-table [id=a07be8cb50d341068546068247605f10]) successfully deleted
W20260811 18:52:03.262140 14554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:42931 (error 108)
I20260811 18:52:03.262283 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 14956
I20260811 18:52:03.309501 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:03.309319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:42931 (local address 127.0.0.1:40808)
0811 18:52:03.309382 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:03.309451 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:42931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 18:52:03.309705 14554 catalog_manager.cc:4584] TS cca4e6dcafbc4cfa81aa9dae0678b1fd (127.9.215.1:42931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f0efe533b1943e58f0664971f85e3d8: Network error: Client connection negotiation failed: client connection to 127.9.215.1:42931: connect: Connection refused (error 111)
I20260811 18:52:03.313390 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15219
I20260811 18:52:03.328466 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 14527
2026-08-11T18:52:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (992 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:03 dist-test-slave-pww0 krb5kdc[15562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:03 dist-test-slave-pww0 krb5kdc[15562](info): set up 2 sockets
Aug 11 18:52:03 dist-test-slave-pww0 krb5kdc[15562](info): commencing operation
krb5kdc: starting...
W20260811 18:52:04.578949 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:04 dist-test-slave-pww0 krb5kdc[15562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:04.711359 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34785
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:34785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:04.798084 15590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:04.798326 15590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:04.798353 15590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:04.799618 15590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:04.799671 15590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:04.799688 15590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:04.799702 15590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:04.799715 15590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:04.801354 15590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:34785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:34785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:04.801645 15590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:04.801893 15590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:04.807129 15596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:04.807120 15601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:04.807120 15606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:04.807236 15590 server_base.cc:1034] running on GCE node
I20260811 18:52:04.807709 15590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:04.808015 15590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:04.809180 15590 hybrid_clock.cc:648] HybridClock initialized: now 1786474324809165 us; error 44 us; skew 500 ppm
Aug 11 18:52:04 dist-test-slave-pww0 krb5kdc[15562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474324, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:04.811901 15590 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:04.812413 15590 webserver.cc:460] Webserver started at http://127.9.215.62:38747/ using document root <none> and password file <none>
I20260811 18:52:04.812655 15590 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:04.812708 15590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:04.812847 15590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:04.813983 15590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "70a946e7d79c413c86e66a0e7f611d32"
format_stamp: "Formatted at 2026-08-11 18:52:04 on dist-test-slave-pww0"
server_key: "c64ac858e41cfa97f7be2a8fe6355fdb"
server_key_iv: "d03e0d0fd8bc9e507eecd487decd6ec5"
server_key_version: "encryptionkey@0"
I20260811 18:52:04.814368 15590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "70a946e7d79c413c86e66a0e7f611d32"
format_stamp: "Formatted at 2026-08-11 18:52:04 on dist-test-slave-pww0"
server_key: "c64ac858e41cfa97f7be2a8fe6355fdb"
server_key_iv: "d03e0d0fd8bc9e507eecd487decd6ec5"
server_key_version: "encryptionkey@0"
I20260811 18:52:04.815927 15590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:04.816857 15617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:04.817095 15590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:04.817199 15590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "70a946e7d79c413c86e66a0e7f611d32"
format_stamp: "Formatted at 2026-08-11 18:52:04 on dist-test-slave-pww0"
server_key: "c64ac858e41cfa97f7be2a8fe6355fdb"
server_key_iv: "d03e0d0fd8bc9e507eecd487decd6ec5"
server_key_version: "encryptionkey@0"
I20260811 18:52:04.817274 15590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:04.830684 15590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:04.833299 15590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:04.833469 15590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:04.837714 15590 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:34785
I20260811 18:52:04.837884 15722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:34785 every 8 connection(s)
I20260811 18:52:04.838125 15590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:04.839089 15724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:04.841589 15724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32: Bootstrap starting.
I20260811 18:52:04.842195 15724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:04.842483 15724 log.cc:824] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:04.843222 15724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32: No bootstrap required, opened a new log
I20260811 18:52:04.844915 15724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a946e7d79c413c86e66a0e7f611d32" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34785 } }
I20260811 18:52:04.845060 15724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:04.845082 15724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a946e7d79c413c86e66a0e7f611d32, State: Initialized, Role: FOLLOWER
I20260811 18:52:04.845229 15724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a946e7d79c413c86e66a0e7f611d32" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34785 } }
I20260811 18:52:04.845302 15724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:04.845324 15724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:04.845373 15724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:04.845930 15724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a946e7d79c413c86e66a0e7f611d32" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34785 } }
I20260811 18:52:04.846021 15724 leader_election.cc:302] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70a946e7d79c413c86e66a0e7f611d32; no voters:
I20260811 18:52:04.846221 15724 leader_election.cc:288] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:04.846282 15730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:04.846462 15724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:04.846479 15730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [term 1 LEADER]: Becoming Leader. State: Replica: 70a946e7d79c413c86e66a0e7f611d32, State: Running, Role: LEADER
I20260811 18:52:04.846663 15730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a946e7d79c413c86e66a0e7f611d32" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34785 } }
I20260811 18:52:04.847298 15733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a946e7d79c413c86e66a0e7f611d32. Latest consensus state: current_term: 1 leader_uuid: "70a946e7d79c413c86e66a0e7f611d32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a946e7d79c413c86e66a0e7f611d32" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34785 } } }
I20260811 18:52:04.847481 15733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:04.847496 15732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70a946e7d79c413c86e66a0e7f611d32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a946e7d79c413c86e66a0e7f611d32" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 34785 } } }
I20260811 18:52:04.847595 15732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:04.847663 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15590
I20260811 18:52:04.847774 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:04.847946 10076 external_mini_cluster.cc:1428] Setting key ec60e272ce36d0bddd9400a5cc1f75f1
I20260811 18:52:04.848042 15741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:04.848570 15741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:04.850191 15741 catalog_manager.cc:1347] Generated new cluster ID: faa7680615904459b98cdc935d69f8e0
I20260811 18:52:04.850248 15741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:52:04 dist-test-slave-pww0 krb5kdc[15562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:04.865708 15741 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:04.866385 15741 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:04.869094 15741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32: Generated new TSK 0
I20260811 18:52:04.869279 15741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:04.869360 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:04.848439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.9.215.62:34785)
0811 18:52:04.848614 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:04.848626 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:04.856915 (+ 8289us) server_negotiation.cc:407] Connection header received
0811 18:52:04.857093 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:04.857101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:04.857167 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:04.857330 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:04.858149 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:04.859649 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:04.860766 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:04.861432 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:04.861529 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:04.864684 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:04.864689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:04.864690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:04.864705 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:04.867063 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:04.867516 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:04.867520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:04.867520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:04.867673 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:04.867895 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:04.867897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:04.867897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:04.868239 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:04.868404 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:04.868735 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 18:52:04.868911 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.9.215.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
Entry for principal kudu/127.9.215.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab.
I20260811 18:52:04.899513 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34785
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:04.989004 15766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:04.989275 15766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:04.989315 15766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:04.990625 15766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:04.990686 15766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:04.990731 15766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:52:04.992404 15766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.1.keytab
--principal=kudu/127.9.215.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:04.992697 15766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:04.992949 15766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:04.998555 15775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:04.998557 15780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:04.998585 15778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:04.998633 15766 server_base.cc:1034] running on GCE node
I20260811 18:52:04.999099 15766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:04.999356 15766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:05.000530 15766 hybrid_clock.cc:648] HybridClock initialized: now 1786474325000486 us; error 85 us; skew 500 ppm
Aug 11 18:52:04 dist-test-slave-pww0 krb5kdc[15562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474324, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:05.003403 15766 init.cc:377] Logged in from keytab as kudu/127.9.215.1@KRBTEST.COM (short username kudu)
I20260811 18:52:05.003978 15766 webserver.cc:460] Webserver started at http://127.9.215.1:35631/ using document root <none> and password file <none>
I20260811 18:52:05.004208 15766 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:05.004264 15766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:05.004402 15766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:05.005673 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "2b29d4d57c754260899b683e0f7611ae"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "199cdcd9e446289fa072474f59a6a67a"
server_key_iv: "588308be8b9b29170d22ec233c5e1860"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.006059 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "2b29d4d57c754260899b683e0f7611ae"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "199cdcd9e446289fa072474f59a6a67a"
server_key_iv: "588308be8b9b29170d22ec233c5e1860"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.007540 15766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:05.008411 15792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.008627 15766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:05.008745 15766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "2b29d4d57c754260899b683e0f7611ae"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "199cdcd9e446289fa072474f59a6a67a"
server_key_iv: "588308be8b9b29170d22ec233c5e1860"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.008829 15766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:05.029567 15766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:05.031939 15766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:05.032106 15766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:05.032358 15766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:05.032677 15766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:05.032712 15766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.032743 15766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:05.032768 15766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.038688 15766 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:43179
I20260811 18:52:05.038780 16037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:43179 every 8 connection(s)
I20260811 18:52:05.039103 15766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
Aug 11 18:52:05 dist-test-slave-pww0 krb5kdc[15562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474324, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.1@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:05.046339 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 15766
I20260811 18:52:05.046465 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:52:05.046675 10076 external_mini_cluster.cc:1428] Setting key 33b6f6f3ce6c02b58a586d65738c8c50
I20260811 18:52:05.050051 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.039944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:47559 (local address 127.9.215.62:34785)
0811 18:52:05.040021 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.040023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.040271 (+ 248us) server_negotiation.cc:407] Connection header received
0811 18:52:05.041242 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.041243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.041254 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.041318 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:05.042110 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.042644 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.043832 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.044054 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.044136 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.046694 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:05.046696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:05.046697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:05.046707 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:05.048309 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.048852 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.048853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.048853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.048921 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.049255 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.049256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.049256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.049443 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:05.049531 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.049829 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.049952 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:52:05.050129 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.039923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34785 (local address 127.9.215.1:47559)
0811 18:52:05.040106 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.040151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.041118 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.041452 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.041457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.041766 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:05.041983 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.041988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.042774 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.042775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.043739 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.043742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.044161 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.044162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.044293 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.044833 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:05.044842 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:05.046540 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:05.048394 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:05.048406 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:05.048409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:05.048749 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:05.048989 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:05.048990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:05.048990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:05.049147 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:05.049525 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:05.049533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:05.049698 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.049791 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.049937 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260811 18:52:05.050711 16038 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34785
I20260811 18:52:05.050817 16038 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:05.051011 16038 heartbeater.cc:511] Master 127.9.215.62:34785 requested a full tablet report, sending...
I20260811 18:52:05.051653 15653 ts_manager.cc:194] Registered new tserver with Master: 2b29d4d57c754260899b683e0f7611ae (127.9.215.1:43179)
I20260811 18:52:05.052575 15653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.1@KRBTEST.COM'} at 127.9.215.1:47559
WARNING: no policy specified for kudu/127.9.215.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
Entry for principal kudu/127.9.215.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab.
I20260811 18:52:05.077780 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34785
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:05.168589 16048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:05.168825 16048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:05.168850 16048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:05.170074 16048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:05.170131 16048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:05.170159 16048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:52:05.171801 16048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.2.keytab
--principal=kudu/127.9.215.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:05.172058 16048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:05.172302 16048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:05.177867 16058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:05.177883 16059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:05.177884 16063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:05.178174 16048 server_base.cc:1034] running on GCE node
I20260811 18:52:05.178424 16048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:05.178689 16048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:05.179829 16048 hybrid_clock.cc:648] HybridClock initialized: now 1786474325179814 us; error 36 us; skew 500 ppm
Aug 11 18:52:05 dist-test-slave-pww0 krb5kdc[15562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474325, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:05.182430 16048 init.cc:377] Logged in from keytab as kudu/127.9.215.2@KRBTEST.COM (short username kudu)
I20260811 18:52:05.182899 16048 webserver.cc:460] Webserver started at http://127.9.215.2:45623/ using document root <none> and password file <none>
I20260811 18:52:05.183105 16048 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:05.183156 16048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:05.183292 16048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:05.184438 16048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "981cf0b599474bd1ac6545b2d1687ce1"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "134e96169fbfb655c836d41df332a649"
server_key_iv: "d69ec67c9a583f6d1808deb67bd95f9b"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.184779 16048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "981cf0b599474bd1ac6545b2d1687ce1"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "134e96169fbfb655c836d41df332a649"
server_key_iv: "d69ec67c9a583f6d1808deb67bd95f9b"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.186084 16048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:05.187047 16079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.187278 16048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:05.187372 16048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "981cf0b599474bd1ac6545b2d1687ce1"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "134e96169fbfb655c836d41df332a649"
server_key_iv: "d69ec67c9a583f6d1808deb67bd95f9b"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.187441 16048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:05.195359 16048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:05.197788 16048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:05.197953 16048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:05.198218 16048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:05.198570 16048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:05.198606 16048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.198637 16048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:05.198662 16048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.205408 16048 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:35603
I20260811 18:52:05.205492 16304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:35603 every 8 connection(s)
I20260811 18:52:05.205869 16048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:05.206836 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16048
I20260811 18:52:05.206962 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:52:05.207175 10076 external_mini_cluster.cc:1428] Setting key 3964bc3cb5959c7fe21cfe37d9188c63
Aug 11 18:52:05 dist-test-slave-pww0 krb5kdc[15562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474325, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.2@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:05.217092 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.206650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:39121 (local address 127.9.215.62:34785)
0811 18:52:05.206725 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.206727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.207020 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:52:05.207681 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.207682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.207696 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.207786 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:05.209084 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.209626 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.210844 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.211050 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.211111 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.213845 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:05.213847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:05.213848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:05.213858 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:05.215603 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.216081 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.216083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.216083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.216152 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.216396 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.216397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.216398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.216554 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:05.216644 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.216891 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.216994 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.9.215.3@KRBTEST.COM; defaulting to no policy
I20260811 18:52:05.217213 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.206606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34785 (local address 127.9.215.2:39121)
0811 18:52:05.206804 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.206865 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.207565 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.207846 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.207850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.208636 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:05.208959 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.208967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.209739 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.209740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.210739 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.210742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.211135 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.211136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.211282 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.211929 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:05.211939 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:05.213713 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:05.215713 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:05.215724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:05.215727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:05.215997 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:05.216214 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:05.216221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:05.216222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:05.216336 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:05.216616 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:05.216621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:05.216765 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.216850 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.216993 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260811 18:52:05.217747 16306 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34785
I20260811 18:52:05.217855 16306 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:05.218032 16306 heartbeater.cc:511] Master 127.9.215.62:34785 requested a full tablet report, sending...
I20260811 18:52:05.218502 15653 ts_manager.cc:194] Registered new tserver with Master: 981cf0b599474bd1ac6545b2d1687ce1 (127.9.215.2:35603)
I20260811 18:52:05.218907 15653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.2@KRBTEST.COM'} at 127.9.215.2:39121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
Entry for principal kudu/127.9.215.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab.
I20260811 18:52:05.239063 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34785
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:05.324038 16323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:05.324282 16323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:05.324308 16323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:05.325525 16323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:05.325572 16323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:05.325599 16323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:52:05.327190 16323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.3.keytab
--principal=kudu/127.9.215.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:34785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:05.327416 16323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:05.327687 16323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:05.333356 16338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:05.333349 16340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:05.333391 16323 server_base.cc:1034] running on GCE node
W20260811 18:52:05.333350 16342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:05.333766 16323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:05.334069 16323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:05.335238 16323 hybrid_clock.cc:648] HybridClock initialized: now 1786474325335214 us; error 30 us; skew 500 ppm
Aug 11 18:52:05 dist-test-slave-pww0 krb5kdc[15562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474325, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:05.338361 16323 init.cc:377] Logged in from keytab as kudu/127.9.215.3@KRBTEST.COM (short username kudu)
I20260811 18:52:05.338907 16323 webserver.cc:460] Webserver started at http://127.9.215.3:36089/ using document root <none> and password file <none>
I20260811 18:52:05.339133 16323 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:05.339191 16323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:05.339341 16323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:05.340618 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "42706016d4ac4f238cd1767b6b0bd8d6"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "868be4ec2cd96a4d203c8fddce04973b"
server_key_iv: "29bc28ab4676e9cf9b2d8ddc0e514902"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.340989 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "42706016d4ac4f238cd1767b6b0bd8d6"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "868be4ec2cd96a4d203c8fddce04973b"
server_key_iv: "29bc28ab4676e9cf9b2d8ddc0e514902"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.342451 16323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:05.343387 16353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.343613 16323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:05.343716 16323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "42706016d4ac4f238cd1767b6b0bd8d6"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "868be4ec2cd96a4d203c8fddce04973b"
server_key_iv: "29bc28ab4676e9cf9b2d8ddc0e514902"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.343794 16323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:05.363778 16323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:05.366259 16323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:05.366420 16323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:05.366668 16323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:05.367097 16323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:05.367139 16323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.367170 16323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:05.367187 16323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.373304 16323 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:43015
I20260811 18:52:05.373368 16590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:43015 every 8 connection(s)
I20260811 18:52:05.373694 16323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:05.375438 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16323
I20260811 18:52:05.375623 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:52:05.375818 10076 external_mini_cluster.cc:1428] Setting key aca1cec606f340670a16a5f7e42ebd11
Aug 11 18:52:05 dist-test-slave-pww0 krb5kdc[15562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474325, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.3@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:05.384392 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.374578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:42943 (local address 127.9.215.62:34785)
0811 18:52:05.374645 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.374647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.374819 (+ 172us) server_negotiation.cc:407] Connection header received
0811 18:52:05.375574 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.375576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.375595 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.375677 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:05.376990 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.377550 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.378686 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.378892 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.378961 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.381395 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:05.381397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:05.381397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:05.381405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:05.382796 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.383253 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.383254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.383255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.383316 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.383671 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.383671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.383672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.383795 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:05.383850 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.384222 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.384318 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 18:52:05.384613 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.374469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:34785 (local address 127.9.215.3:42943)
0811 18:52:05.374660 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.374706 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.375211 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.375827 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.375832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.376603 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:05.376850 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.376858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.377706 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.377707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.378579 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.378581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.379001 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.379002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.379159 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.379796 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:05.379806 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:05.381292 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:05.382888 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:05.382898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:05.382901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:05.383173 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:05.383413 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:05.383414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:05.383415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:05.383605 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:05.383895 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:05.383905 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:05.384060 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.384171 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.384330 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260811 18:52:05.385123 16593 heartbeater.cc:347] Connected to a master server at 127.9.215.62:34785
I20260811 18:52:05.385231 16593 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:05.385433 16593 heartbeater.cc:511] Master 127.9.215.62:34785 requested a full tablet report, sending...
I20260811 18:52:05.385872 15653 ts_manager.cc:194] Registered new tserver with Master: 42706016d4ac4f238cd1767b6b0bd8d6 (127.9.215.3:43015)
I20260811 18:52:05.386235 15653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.215.3@KRBTEST.COM'} at 127.9.215.3:42943
I20260811 18:52:05.387941 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:05.394896 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.389242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.9.215.62:34785)
0811 18:52:05.389313 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.389314 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.389397 (+ 83us) server_negotiation.cc:407] Connection header received
0811 18:52:05.389450 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.389451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.389460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.389506 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:05.390021 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.390576 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.391281 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.391506 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.391560 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.392663 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:05.392664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:05.392664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:05.392672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:05.393945 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.394236 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.394237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.394237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.394290 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:05.394409 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:05.394410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:05.394410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:05.394556 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:05.394635 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.394722 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.394799 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:52:05.396713 15653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:05.398362 15653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:05.404878 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.400677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:35603 (local address 127.0.0.1:45150)
0811 18:52:05.401026 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.401037 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.401097 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.401640 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.401640 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.401643 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:05.401672 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.401674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.402606 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.402607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.403660 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.403662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.404544 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.404545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.404628 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.404645 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.404722 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.404793 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":62,"thread_start_us":105,"threads_started":1}
I20260811 18:52:05.404878 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.400803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.9.215.2:35603)
0811 18:52:05.401488 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.401492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.401501 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:05.401537 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.401539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.401552 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.401741 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:05.401808 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.402506 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.403874 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.404416 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.404539 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.404616 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.404740 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.404783 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":82,"threads_started":1}
I20260811 18:52:05.405094 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.400670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.0.0.1:35464)
0811 18:52:05.400869 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.400912 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.400985 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.401181 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.401184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.401187 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:05.401264 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.401266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.402645 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.402646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.403899 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.403902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.404866 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.404867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.404934 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.404939 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.404986 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.405031 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":38,"thread_start_us":83,"threads_started":1}
I20260811 18:52:05.405457 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.400905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:43015 (local address 127.0.0.1:37590)
0811 18:52:05.401303 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.401313 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.401363 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.401610 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.401611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.401613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:05.401672 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.401674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.402902 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.402903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.404277 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.404280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.405218 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.405219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.405285 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.405290 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.405341 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.405383 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":70,"threads_started":1}
I20260811 18:52:05.405510 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.400808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.9.215.1:43179)
0811 18:52:05.401022 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.401028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.401040 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:52:05.401081 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.401083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.401098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.401171 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:05.401330 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.402526 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.404001 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.404719 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.404778 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.404834 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.405289 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.405328 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":94,"threads_started":1}
I20260811 18:52:05.405572 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.401128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.9.215.3:43015)
0811 18:52:05.401296 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.401300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.401355 (+ 55us) server_negotiation.cc:407] Connection header received
0811 18:52:05.401431 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.401433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.401447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.401520 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:05.401745 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.402786 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.404363 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.405064 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.405130 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.405181 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.405441 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.405496 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260811 18:52:05.406104 16176 tablet_service.cc:1467] Processing CreateTablet for tablet 227321799f034d7db05f9749613eec1f (DEFAULT_TABLE table=test-table [id=106feb37af3943efb0321d48d3b17ab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:05.406141 15887 tablet_service.cc:1467] Processing CreateTablet for tablet 227321799f034d7db05f9749613eec1f (DEFAULT_TABLE table=test-table [id=106feb37af3943efb0321d48d3b17ab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:05.406440 15887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 227321799f034d7db05f9749613eec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:05.406441 16176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 227321799f034d7db05f9749613eec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:05.408335 16453 tablet_service.cc:1467] Processing CreateTablet for tablet 227321799f034d7db05f9749613eec1f (DEFAULT_TABLE table=test-table [id=106feb37af3943efb0321d48d3b17ab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:05.408727 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 227321799f034d7db05f9749613eec1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:05.409345 16627 tablet_bootstrap.cc:492] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Bootstrap starting.
I20260811 18:52:05.409713 16629 tablet_bootstrap.cc:492] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae: Bootstrap starting.
I20260811 18:52:05.410113 16627 tablet_bootstrap.cc:654] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:05.410418 16629 tablet_bootstrap.cc:654] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:05.410707 16627 log.cc:824] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:05.410986 16629 log.cc:824] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:05.411624 16627 tablet_bootstrap.cc:492] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: No bootstrap required, opened a new log
I20260811 18:52:05.411687 16627 ts_tablet_manager.cc:1397] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.411897 16629 tablet_bootstrap.cc:492] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae: No bootstrap required, opened a new log
I20260811 18:52:05.411955 16629 ts_tablet_manager.cc:1397] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.413318 16627 raft_consensus.cc:348] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.413453 16627 raft_consensus.cc:374] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:05.413473 16627 raft_consensus.cc:729] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981cf0b599474bd1ac6545b2d1687ce1, State: Initialized, Role: FOLLOWER
I20260811 18:52:05.413564 16627 consensus_queue.cc:260] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.413492 16629 raft_consensus.cc:348] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.413652 16629 raft_consensus.cc:374] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:05.413678 16629 raft_consensus.cc:729] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b29d4d57c754260899b683e0f7611ae, State: Initialized, Role: FOLLOWER
I20260811 18:52:05.413766 16629 consensus_queue.cc:260] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.413805 16627 ts_tablet_manager.cc:1428] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.413863 16306 heartbeater.cc:503] Master 127.9.215.62:34785 was elected leader, sending a full tablet report...
I20260811 18:52:05.414036 16629 ts_tablet_manager.cc:1428] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.414157 16038 heartbeater.cc:503] Master 127.9.215.62:34785 was elected leader, sending a full tablet report...
I20260811 18:52:05.414333 16635 tablet_bootstrap.cc:492] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Bootstrap starting.
I20260811 18:52:05.415069 16635 tablet_bootstrap.cc:654] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:05.415649 16635 log.cc:824] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:05.416426 16635 tablet_bootstrap.cc:492] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: No bootstrap required, opened a new log
I20260811 18:52:05.416494 16635 ts_tablet_manager.cc:1397] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.417987 16635 raft_consensus.cc:348] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.418169 16635 raft_consensus.cc:374] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:05.418192 16635 raft_consensus.cc:729] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42706016d4ac4f238cd1767b6b0bd8d6, State: Initialized, Role: FOLLOWER
I20260811 18:52:05.418318 16635 consensus_queue.cc:260] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.418624 16635 ts_tablet_manager.cc:1428] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.418721 16593 heartbeater.cc:503] Master 127.9.215.62:34785 was elected leader, sending a full tablet report...
I20260811 18:52:05.451102 16640 raft_consensus.cc:482] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:05.451267 16640 raft_consensus.cc:504] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.451756 16640 leader_election.cc:288] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b29d4d57c754260899b683e0f7611ae (127.9.215.1:43179), 981cf0b599474bd1ac6545b2d1687ce1 (127.9.215.2:35603)
I20260811 18:52:05.456076 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.452141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:56603 (local address 127.9.215.1:43179)
0811 18:52:05.452185 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.452188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.452307 (+ 119us) server_negotiation.cc:407] Connection header received
0811 18:52:05.452351 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.452352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.452364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.452416 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:05.452521 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.453235 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.454826 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.455501 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.455583 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.455596 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.455905 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.455961 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:05.456043 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.452056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:44575 (local address 127.9.215.2:35603)
0811 18:52:05.452115 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.452118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.452128 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:05.452160 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.452161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.452176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.452237 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:05.452407 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.453191 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.454826 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.455500 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.455581 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.455596 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.455905 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.455961 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:52:05.456041 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.451965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:35603 (local address 127.9.215.3:44575)
0811 18:52:05.452009 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.452023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.452073 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.452258 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.452259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.452261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:05.452336 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.452338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.453309 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.453311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.454675 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.454680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.455610 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.455612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.455775 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.455782 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.455861 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.455929 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":83}
I20260811 18:52:05.456055 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.451995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.9.215.3:56603)
0811 18:52:05.452232 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.452242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.452298 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.452418 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.452418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.452420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:05.452469 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.452470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.453316 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.453317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.454675 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.454680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.455610 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.455612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.455775 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.455782 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.455861 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.455929 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":67,"thread_start_us":171,"threads_started":1}
W20260811 18:52:05.456502 16311 tablet.cc:2369] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:05.456535 16220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227321799f034d7db05f9749613eec1f" candidate_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981cf0b599474bd1ac6545b2d1687ce1" is_pre_election: true
I20260811 18:52:05.456641 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227321799f034d7db05f9749613eec1f" candidate_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b29d4d57c754260899b683e0f7611ae" is_pre_election: true
I20260811 18:52:05.456725 16220 raft_consensus.cc:2454] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42706016d4ac4f238cd1767b6b0bd8d6 in term 0.
I20260811 18:52:05.456802 15932 raft_consensus.cc:2454] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42706016d4ac4f238cd1767b6b0bd8d6 in term 0.
I20260811 18:52:05.457013 16355 leader_election.cc:302] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42706016d4ac4f238cd1767b6b0bd8d6, 981cf0b599474bd1ac6545b2d1687ce1; no voters:
I20260811 18:52:05.457185 16640 raft_consensus.cc:2781] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:05.457254 16640 raft_consensus.cc:482] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:05.457273 16640 raft_consensus.cc:3037] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:05.457988 16640 raft_consensus.cc:504] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.458118 16640 leader_election.cc:288] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [CANDIDATE]: Term 1 election: Requested vote from peers 2b29d4d57c754260899b683e0f7611ae (127.9.215.1:43179), 981cf0b599474bd1ac6545b2d1687ce1 (127.9.215.2:35603)
I20260811 18:52:05.458333 15932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227321799f034d7db05f9749613eec1f" candidate_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b29d4d57c754260899b683e0f7611ae"
I20260811 18:52:05.458415 15932 raft_consensus.cc:3037] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:05.459223 15932 raft_consensus.cc:2454] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42706016d4ac4f238cd1767b6b0bd8d6 in term 1.
I20260811 18:52:05.459412 16357 leader_election.cc:302] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b29d4d57c754260899b683e0f7611ae, 42706016d4ac4f238cd1767b6b0bd8d6; no voters:
I20260811 18:52:05.459417 16220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "227321799f034d7db05f9749613eec1f" candidate_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "981cf0b599474bd1ac6545b2d1687ce1"
I20260811 18:52:05.459527 16220 raft_consensus.cc:3037] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:05.459586 16640 raft_consensus.cc:2781] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:05.459774 16640 raft_consensus.cc:686] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 1 LEADER]: Becoming Leader. State: Replica: 42706016d4ac4f238cd1767b6b0bd8d6, State: Running, Role: LEADER
I20260811 18:52:05.459895 16640 consensus_queue.cc:237] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } }
I20260811 18:52:05.460382 16220 raft_consensus.cc:2454] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42706016d4ac4f238cd1767b6b0bd8d6 in term 1.
I20260811 18:52:05.460963 15653 catalog_manager.cc:5495] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42706016d4ac4f238cd1767b6b0bd8d6 (127.9.215.3). New cstate: current_term: 1 leader_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42706016d4ac4f238cd1767b6b0bd8d6" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 43015 } health_report { overall_health: HEALTHY } } }
I20260811 18:52:05.478834 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.476472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37594 (local address 127.9.215.3:43015)
0811 18:52:05.476536 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.476538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.476547 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:05.476591 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.476592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.476614 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.476675 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:05.476830 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.477313 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.478063 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.478265 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.478321 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.478485 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:05.478594 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:05.478668 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.478705 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.478728 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:52:05.480880 16220 raft_consensus.cc:1261] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 1 FOLLOWER]: Refusing update from remote peer 42706016d4ac4f238cd1767b6b0bd8d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:05.480885 15932 raft_consensus.cc:1261] T 227321799f034d7db05f9749613eec1f P 2b29d4d57c754260899b683e0f7611ae [term 1 FOLLOWER]: Refusing update from remote peer 42706016d4ac4f238cd1767b6b0bd8d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:05.481240 16640 consensus_queue.cc:1035] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "981cf0b599474bd1ac6545b2d1687ce1" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 35603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:05.481385 16646 consensus_queue.cc:1035] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b29d4d57c754260899b683e0f7611ae" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 43179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:05.483759 16650 mvcc.cc:205] Tried to move back new op lower bound from 7317398837166714880 to 7317398837084561408. Current Snapshot: MvccSnapshot[applied={T|T < 7317398837166714880}]
I20260811 18:52:05.483942 16652 mvcc.cc:205] Tried to move back new op lower bound from 7317398837166714880 to 7317398837084561408. Current Snapshot: MvccSnapshot[applied={T|T < 7317398837166714880}]
I20260811 18:52:05.484330 16651 mvcc.cc:205] Tried to move back new op lower bound from 7317398837166714880 to 7317398837084561408. Current Snapshot: MvccSnapshot[applied={T|T < 7317398837166714880}]
I20260811 18:52:05.486370 15652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:05.486465 15652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:05.487331 15652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70a946e7d79c413c86e66a0e7f611d32: Sending DeleteTablet for 3 replicas of tablet 227321799f034d7db05f9749613eec1f
I20260811 18:52:05.487732 15887 tablet_service.cc:1514] Processing DeleteTablet for tablet 227321799f034d7db05f9749613eec1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:05 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:35464
I20260811 18:52:05.487747 16453 tablet_service.cc:1514] Processing DeleteTablet for tablet 227321799f034d7db05f9749613eec1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:05 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:37590
I20260811 18:52:05.488049 16176 tablet_service.cc:1514] Processing DeleteTablet for tablet 227321799f034d7db05f9749613eec1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:05 UTC) from {username='kudu', principal='kudu/127.9.215.62@KRBTEST.COM'} at 127.0.0.1:45150
I20260811 18:52:05.488294 16678 tablet_replica.cc:330] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: stopping tablet replica
I20260811 18:52:05.488294 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15766
I20260811 18:52:05.488422 16678 raft_consensus.cc:2229] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:05.488624 16678 raft_consensus.cc:2258] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:05.489257 16678 ts_tablet_manager.cc:1905] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:05.491663 16678 ts_tablet_manager.cc:1918] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:05.491851 16678 log.cc:1196] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/227321799f034d7db05f9749613eec1f
I20260811 18:52:05.492267 16678 ts_tablet_manager.cc:1939] T 227321799f034d7db05f9749613eec1f P 981cf0b599474bd1ac6545b2d1687ce1: Deleting consensus metadata
I20260811 18:52:05.499596 16675 tablet_replica.cc:330] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: stopping tablet replica
I20260811 18:52:05.499900 16675 raft_consensus.cc:2229] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:52:05.500079 16675 raft_consensus.cc:2258] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:05.506389 15625 catalog_manager.cc:4841] TS 981cf0b599474bd1ac6545b2d1687ce1 (127.9.215.2:35603): tablet 227321799f034d7db05f9749613eec1f (table test-table [id=106feb37af3943efb0321d48d3b17ab4]) successfully deleted
I20260811 18:52:05.509655 16675 ts_tablet_manager.cc:1905] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:05.511608 16675 ts_tablet_manager.cc:1918] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:05.512071 16675 log.cc:1196] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/227321799f034d7db05f9749613eec1f
I20260811 18:52:05.512440 16675 ts_tablet_manager.cc:1939] T 227321799f034d7db05f9749613eec1f P 42706016d4ac4f238cd1767b6b0bd8d6: Deleting consensus metadata
I20260811 18:52:05.513016 15625 catalog_manager.cc:4841] TS 42706016d4ac4f238cd1767b6b0bd8d6 (127.9.215.3:43015): tablet 227321799f034d7db05f9749613eec1f (table test-table [id=106feb37af3943efb0321d48d3b17ab4]) successfully deleted
W20260811 18:52:05.513700 15633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:43179 (error 108)
I20260811 18:52:05.513967 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16048
I20260811 18:52:05.519608 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.517383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:43179 (local address 127.0.0.1:35470)
0811 18:52:05.519442 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.519553 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2027}
W20260811 18:52:05.522365 15633 catalog_manager.cc:4584] TS 2b29d4d57c754260899b683e0f7611ae (127.9.215.1:43179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 227321799f034d7db05f9749613eec1f: Network error: Client connection negotiation failed: client connection to 127.9.215.1:43179: connect: Connection refused (error 111)
I20260811 18:52:05.545569 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16323
I20260811 18:52:05.560990 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 15590
2026-08-11T18:52:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2242 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T18:52:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:05Z Disabled control of system clock
I20260811 18:52:05.608002 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:44567
--webserver_interface=127.9.215.62
--webserver_port=0
--builtin_ntp_servers=127.9.215.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:44567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:05.692749 16686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:05.693012 16686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:05.693044 16686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:05.694362 16686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:05.694427 16686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:05.694443 16686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:05.694460 16686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:05.694475 16686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:05.696167 16686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:44567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:44567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:05.696498 16686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:05.696827 16686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:05.702219 16694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:05.702356 16686 server_base.cc:1034] running on GCE node
W20260811 18:52:05.702491 16695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:05.702595 16702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:05.702932 16686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:05.703301 16686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:05.704501 16686 hybrid_clock.cc:648] HybridClock initialized: now 1786474325704463 us; error 49 us; skew 500 ppm
I20260811 18:52:05.706022 16686 webserver.cc:460] Webserver started at http://127.9.215.62:44283/ using document root <none> and password file <none>
I20260811 18:52:05.706274 16686 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:05.706331 16686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:05.706473 16686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:05.707796 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "b6401b2231387257318da3bba46ae3a7"
server_key_iv: "6b9a6712ce081278f38081e208717719"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.708199 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "b6401b2231387257318da3bba46ae3a7"
server_key_iv: "6b9a6712ce081278f38081e208717719"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.709967 16686 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:52:05.710961 16719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.711205 16686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:05.711324 16686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "b6401b2231387257318da3bba46ae3a7"
server_key_iv: "6b9a6712ce081278f38081e208717719"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.711400 16686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:05.742472 16686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:05.742851 16686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:05.743010 16686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:05.747448 16686 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:44567
I20260811 18:52:05.747525 16827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:44567 every 8 connection(s)
I20260811 18:52:05.747890 16686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:05.748910 16829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:05.751514 16829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: Bootstrap starting.
I20260811 18:52:05.752238 16829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:05.752604 16829 log.cc:824] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:05.753336 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16686
I20260811 18:52:05.753461 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:05.753549 16829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: No bootstrap required, opened a new log
I20260811 18:52:05.753679 10076 external_mini_cluster.cc:1428] Setting key 9c6a31081b12587d1ba789918e40c98d
I20260811 18:52:05.755784 16829 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44567 } }
I20260811 18:52:05.755944 16829 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:05.755966 16829 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae31472f05c64dc1b20a6ccea7dd4c7f, State: Initialized, Role: FOLLOWER
I20260811 18:52:05.756078 16829 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44567 } }
I20260811 18:52:05.756166 16829 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:05.756194 16829 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:05.756243 16829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:05.756898 16829 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44567 } }
I20260811 18:52:05.757014 16829 leader_election.cc:302] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae31472f05c64dc1b20a6ccea7dd4c7f; no voters:
I20260811 18:52:05.757208 16829 leader_election.cc:288] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:05.757262 16834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:05.757441 16834 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [term 1 LEADER]: Becoming Leader. State: Replica: ae31472f05c64dc1b20a6ccea7dd4c7f, State: Running, Role: LEADER
I20260811 18:52:05.757511 16829 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:05.757624 16834 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44567 } }
I20260811 18:52:05.757989 16834 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae31472f05c64dc1b20a6ccea7dd4c7f. Latest consensus state: current_term: 1 leader_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44567 } } }
I20260811 18:52:05.758255 16834 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:05.758172 16838 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae31472f05c64dc1b20a6ccea7dd4c7f" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44567 } } }
I20260811 18:52:05.758688 16838 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:05.759626 16874 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:05.759729 16874 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:05.759831 16861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:05.760052 16861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:05.760425 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.754253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.9.215.62:44567)
0811 18:52:05.754538 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.754546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.754576 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:52:05.755013 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.755021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.755086 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.755231 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:05.755504 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.756919 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.758104 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.758779 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.758901 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.759744 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:05.759749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:05.759749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:05.759917 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:05.759958 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:05.759969 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:05.760028 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.760110 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.760198 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":149,"threads_started":1}
I20260811 18:52:05.761914 16861 catalog_manager.cc:1347] Generated new cluster ID: 4aedc36da0784ebf9694bff973ad576f
I20260811 18:52:05.761958 16861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:05.775864 16861 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:05.776495 16861 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:05.780683 16861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: Generated new TSK 0
I20260811 18:52:05.780915 16861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:05.789458 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.1:0
--local_ip_for_outbound_sockets=127.9.215.1
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44567
--builtin_ntp_servers=127.9.215.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:05.884492 16878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:05.884722 16878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:05.884747 16878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:05.886010 16878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:05.886061 16878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:05.886089 16878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.1
I20260811 18:52:05.887611 16878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.215.1
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.1
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:05.888024 16878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:05.888363 16878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:05.894652 16887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:05.894652 16889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:05.894655 16893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:05.895316 16878 server_base.cc:1034] running on GCE node
I20260811 18:52:05.895630 16878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:05.895938 16878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:05.897140 16878 hybrid_clock.cc:648] HybridClock initialized: now 1786474325897120 us; error 36 us; skew 500 ppm
I20260811 18:52:05.898699 16878 webserver.cc:460] Webserver started at http://127.9.215.1:35479/ using document root <none> and password file <none>
I20260811 18:52:05.898978 16878 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:05.899060 16878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:05.899214 16878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:05.900463 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data/instance:
uuid: "23eb5ca11e1747a28ea568e769512dea"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "5927d731e6460fbdcdc0a641289f8e23"
server_key_iv: "b465eb0b3f459008e550d79ead9003f1"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.900858 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance:
uuid: "23eb5ca11e1747a28ea568e769512dea"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "5927d731e6460fbdcdc0a641289f8e23"
server_key_iv: "b465eb0b3f459008e550d79ead9003f1"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.902307 16878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:52:05.903192 16904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.903389 16878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:05.903530 16878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal
uuid: "23eb5ca11e1747a28ea568e769512dea"
format_stamp: "Formatted at 2026-08-11 18:52:05 on dist-test-slave-pww0"
server_key: "5927d731e6460fbdcdc0a641289f8e23"
server_key_iv: "b465eb0b3f459008e550d79ead9003f1"
server_key_version: "encryptionkey@0"
I20260811 18:52:05.903609 16878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:05.920507 16878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:05.920866 16878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:05.921005 16878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:05.921254 16878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:05.921618 16878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:05.921662 16878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.921693 16878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:05.921710 16878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:05.928045 16878 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.1:33933
I20260811 18:52:05.928102 17125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.1:33933 every 8 connection(s)
I20260811 18:52:05.928453 16878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/data/info.pb
I20260811 18:52:05.934259 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.929202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:36199 (local address 127.9.215.62:44567)
0811 18:52:05.929288 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:05.929291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:05.929511 (+ 220us) server_negotiation.cc:407] Connection header received
0811 18:52:05.930018 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:05.930019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:05.930035 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:05.930079 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:05.930804 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.931316 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.932527 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.932759 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.932836 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.933891 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:05.933893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:05.933893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:05.933900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:05.933917 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:05.933921 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:05.933974 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:05.934153 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 18:52:05.934187 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:52:05.934496 17133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:05.929170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44567 (local address 127.9.215.1:36199)
0811 18:52:05.929360 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:05.929399 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:05.929919 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:05.930143 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:05.930147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:05.930155 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:05.930731 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.930738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.931483 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.931485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.932374 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:05.932378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:05.932900 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:05.932901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:05.933060 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:05.933710 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:52:05.933710 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:05.933746 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:52:05.933776 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:05.934022 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:05.934032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:05.934035 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:05.934121 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 18:52:05.934237 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260811 18:52:05.935192 17126 heartbeater.cc:347] Connected to a master server at 127.9.215.62:44567
I20260811 18:52:05.935331 17126 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:05.935597 17126 heartbeater.cc:511] Master 127.9.215.62:44567 requested a full tablet report, sending...
I20260811 18:52:05.936239 16752 ts_manager.cc:194] Registered new tserver with Master: 23eb5ca11e1747a28ea568e769512dea (127.9.215.1:33933)
I20260811 18:52:05.936592 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 16878
I20260811 18:52:05.936692 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-0/wal/instance
I20260811 18:52:05.936882 10076 external_mini_cluster.cc:1428] Setting key 730dfd1bcc6c2597e7ea8c6b02b5a409
I20260811 18:52:05.937145 16752 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.1:36199
I20260811 18:52:05.938144 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.2:0
--local_ip_for_outbound_sockets=127.9.215.2
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44567
--builtin_ntp_servers=127.9.215.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:06.026460 17134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:06.026687 17134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:06.026710 17134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:06.027983 17134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:06.028033 17134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:06.028059 17134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.2
I20260811 18:52:06.029446 17134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.215.2
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.2
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:06.029713 17134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:06.030019 17134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:06.035228 17150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:06.035275 17156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:06.035243 17149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:06.035523 17134 server_base.cc:1034] running on GCE node
I20260811 18:52:06.035730 17134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:06.036026 17134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:06.037212 17134 hybrid_clock.cc:648] HybridClock initialized: now 1786474326037198 us; error 34 us; skew 500 ppm
I20260811 18:52:06.038718 17134 webserver.cc:460] Webserver started at http://127.9.215.2:41449/ using document root <none> and password file <none>
I20260811 18:52:06.038971 17134 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:06.039062 17134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:06.039214 17134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:06.040455 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data/instance:
uuid: "03adb4ebd0f247dcb07c1343cef5029c"
format_stamp: "Formatted at 2026-08-11 18:52:06 on dist-test-slave-pww0"
server_key: "8b8219b249f27fe72de6d261039c6a84"
server_key_iv: "9f1d4db469a079d52445e84e4973885e"
server_key_version: "encryptionkey@0"
I20260811 18:52:06.040843 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance:
uuid: "03adb4ebd0f247dcb07c1343cef5029c"
format_stamp: "Formatted at 2026-08-11 18:52:06 on dist-test-slave-pww0"
server_key: "8b8219b249f27fe72de6d261039c6a84"
server_key_iv: "9f1d4db469a079d52445e84e4973885e"
server_key_version: "encryptionkey@0"
I20260811 18:52:06.042322 17134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:06.043104 17169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:06.043283 17134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 18:52:06.043378 17134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal
uuid: "03adb4ebd0f247dcb07c1343cef5029c"
format_stamp: "Formatted at 2026-08-11 18:52:06 on dist-test-slave-pww0"
server_key: "8b8219b249f27fe72de6d261039c6a84"
server_key_iv: "9f1d4db469a079d52445e84e4973885e"
server_key_version: "encryptionkey@0"
I20260811 18:52:06.043452 17134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:06.058480 17134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:06.058848 17134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:06.058990 17134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:06.059248 17134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:06.059636 17134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:06.059674 17134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:06.059706 17134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:06.059724 17134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:06.065500 17134 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.2:33253
I20260811 18:52:06.065636 17405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.2:33253 every 8 connection(s)
I20260811 18:52:06.065910 17134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/data/info.pb
I20260811 18:52:06.071102 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.066688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.2:47873 (local address 127.9.215.62:44567)
0811 18:52:06.066748 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.066749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.066980 (+ 231us) server_negotiation.cc:407] Connection header received
0811 18:52:06.067530 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.067532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.067544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.067616 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:06.068319 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.068835 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.069794 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.069978 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.070024 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.070862 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:06.070863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:06.070864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:06.070868 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:06.070883 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:06.070887 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:06.070937 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.071001 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.071024 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:52:06.071267 17410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.066652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44567 (local address 127.9.215.2:47873)
0811 18:52:06.066841 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.066884 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.067375 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.067682 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.067685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.067693 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:06.068221 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.068226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.068911 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.068911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.069726 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.069729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.070038 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.070039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.070147 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.070700 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:52:06.070700 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:06.070740 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:52:06.070763 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:06.070943 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:06.070946 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:06.070948 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.070993 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.071076 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260811 18:52:06.071762 17407 heartbeater.cc:347] Connected to a master server at 127.9.215.62:44567
I20260811 18:52:06.071866 17407 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:06.072052 17407 heartbeater.cc:511] Master 127.9.215.62:44567 requested a full tablet report, sending...
I20260811 18:52:06.072436 16750 ts_manager.cc:194] Registered new tserver with Master: 03adb4ebd0f247dcb07c1343cef5029c (127.9.215.2:33253)
I20260811 18:52:06.072818 16750 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.2:47873
I20260811 18:52:06.074710 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17134
I20260811 18:52:06.074812 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal/instance
I20260811 18:52:06.075028 10076 external_mini_cluster.cc:1428] Setting key a1a8339863d855cd07ccf84b29b640ae
I20260811 18:52:06.076081 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.215.3:0
--local_ip_for_outbound_sockets=127.9.215.3
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44567
--builtin_ntp_servers=127.9.215.20:35661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:52:06.161774 17411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:06.161999 17411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:06.162024 17411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:06.163321 17411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:06.163378 17411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:06.163407 17411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.215.3
I20260811 18:52:06.165158 17411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:35661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.215.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.215.3
--webserver_port=0
--tserver_master_addrs=127.9.215.62:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.215.3
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:06.165477 17411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:06.165807 17411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:06.171025 17425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:06.171021 17429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:06.171031 17427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:06.171303 17411 server_base.cc:1034] running on GCE node
I20260811 18:52:06.171577 17411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:06.171881 17411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:06.173064 17411 hybrid_clock.cc:648] HybridClock initialized: now 1786474326173044 us; error 31 us; skew 500 ppm
I20260811 18:52:06.174510 17411 webserver.cc:460] Webserver started at http://127.9.215.3:39091/ using document root <none> and password file <none>
I20260811 18:52:06.174793 17411 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:06.174850 17411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:06.174984 17411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:06.176190 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data/instance:
uuid: "cb413e7858f14cc5882ec54287977df5"
format_stamp: "Formatted at 2026-08-11 18:52:06 on dist-test-slave-pww0"
server_key: "d5ba75b6d41912748da90ad786d1a926"
server_key_iv: "05aec18ad871f837cee9b54f25f95b37"
server_key_version: "encryptionkey@0"
I20260811 18:52:06.176558 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance:
uuid: "cb413e7858f14cc5882ec54287977df5"
format_stamp: "Formatted at 2026-08-11 18:52:06 on dist-test-slave-pww0"
server_key: "d5ba75b6d41912748da90ad786d1a926"
server_key_iv: "05aec18ad871f837cee9b54f25f95b37"
server_key_version: "encryptionkey@0"
I20260811 18:52:06.177937 17411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:06.178817 17438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:06.179023 17411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:06.179119 17411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal
uuid: "cb413e7858f14cc5882ec54287977df5"
format_stamp: "Formatted at 2026-08-11 18:52:06 on dist-test-slave-pww0"
server_key: "d5ba75b6d41912748da90ad786d1a926"
server_key_iv: "05aec18ad871f837cee9b54f25f95b37"
server_key_version: "encryptionkey@0"
I20260811 18:52:06.179209 17411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:06.188759 17411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:06.189090 17411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:06.189222 17411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:06.189460 17411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:52:06.189802 17411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:52:06.189837 17411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:06.189860 17411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:52:06.189884 17411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:06.195590 17411 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.3:36787
I20260811 18:52:06.195660 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.3:36787 every 8 connection(s)
I20260811 18:52:06.195992 17411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/data/info.pb
I20260811 18:52:06.201303 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.196765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.3:44609 (local address 127.9.215.62:44567)
0811 18:52:06.196820 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.196822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.197062 (+ 240us) server_negotiation.cc:407] Connection header received
0811 18:52:06.197598 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.197599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.197612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.197665 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:06.198396 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.198894 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.199931 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.200125 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.200183 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.201029 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:06.201030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:06.201031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:06.201036 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:06.201051 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:06.201056 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:06.201098 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.201182 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.201214 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:52:06.201531 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.196752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44567 (local address 127.9.215.3:44609)
0811 18:52:06.196944 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.196979 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.197497 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.197742 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.197747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.197755 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:06.198307 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.198313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.198987 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.198988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.199844 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.199847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.200212 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.200213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.200344 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.200886 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:52:06.200886 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:06.200917 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:52:06.200942 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:06.201105 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:06.201110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:06.201114 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.201179 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.201301 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260811 18:52:06.202097 17666 heartbeater.cc:347] Connected to a master server at 127.9.215.62:44567
I20260811 18:52:06.202184 17666 heartbeater.cc:464] Registering TS with master...
I20260811 18:52:06.202270 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17411
I20260811 18:52:06.202378 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal/instance
I20260811 18:52:06.202409 17666 heartbeater.cc:511] Master 127.9.215.62:44567 requested a full tablet report, sending...
I20260811 18:52:06.202577 10076 external_mini_cluster.cc:1428] Setting key ff905f9cfe33385ea78320fdacfb830c
I20260811 18:52:06.202870 16750 ts_manager.cc:194] Registered new tserver with Master: cb413e7858f14cc5882ec54287977df5 (127.9.215.3:36787)
I20260811 18:52:06.203308 16750 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.215.3:44609
I20260811 18:52:06.203565 10076 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:52:06.207881 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.205027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.9.215.62:44567)
0811 18:52:06.205073 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.205075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.205228 (+ 153us) server_negotiation.cc:407] Connection header received
0811 18:52:06.205252 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.205252 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.205261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.205299 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:06.205458 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.205932 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.206701 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.206903 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.206958 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.207654 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:06.207655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:06.207655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:06.207660 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:06.207675 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:06.207679 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:06.207742 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.207778 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.207801 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 18:52:06.209568 16750 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:52:06.211221 16750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:52:06.218122 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.213799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33018 (local address 127.9.215.2:33253)
0811 18:52:06.214034 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.214039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.214053 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:52:06.214096 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.214099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.214115 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.214203 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:06.214408 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.215629 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.216987 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.217682 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.217762 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.217870 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.217985 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.218029 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
I20260811 18:52:06.218125 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.213647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:33253 (local address 127.0.0.1:33018)
0811 18:52:06.213859 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.213906 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.213977 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.214213 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.214217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.214220 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:06.214318 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.214321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.215745 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.215747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.216897 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.216900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.217780 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.217781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.217869 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.217883 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.217959 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.218022 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":37,"thread_start_us":80,"threads_started":1}
I20260811 18:52:06.218225 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.213670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:33933 (local address 127.0.0.1:35018)
0811 18:52:06.214073 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.214081 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.214120 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.214625 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.214626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.214628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:06.214673 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.214674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.215956 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.215957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.217114 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.217116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.218020 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.218021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.218084 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.218089 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.218117 (+ 28us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.218159 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":5,"thread_start_us":63,"threads_started":1}
I20260811 18:52:06.218549 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.213799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.9.215.1:33933)
0811 18:52:06.214457 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.214461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.214471 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:06.214508 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.214510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.214525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.214747 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:06.214817 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.215881 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.217207 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.217923 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.218325 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.218405 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.218427 (+ 22us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.218467 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":105,"threads_started":1}
I20260811 18:52:06.218143 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.213872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36787 (local address 127.0.0.1:46412)
0811 18:52:06.214308 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.214317 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.214369 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.214680 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.214681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.214684 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:06.214734 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.214736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.215825 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.215826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.217113 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.217116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.217947 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.217948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.217999 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.218002 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.218043 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.218075 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":64,"thread_start_us":86,"threads_started":1}
I20260811 18:52:06.218963 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.214110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.9.215.3:36787)
0811 18:52:06.214273 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.214277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.214361 (+ 84us) server_negotiation.cc:407] Connection header received
0811 18:52:06.214509 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.214512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.214528 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.214601 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:06.214811 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.215750 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.217228 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.217745 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.217814 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.217913 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.218186 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.218879 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260811 18:52:06.219120 17248 tablet_service.cc:1467] Processing CreateTablet for tablet ff8e584259fb4d2cbe80666d006c7745 (DEFAULT_TABLE table=test-table [id=d2b1708104e64200a76e248b95883276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:06.219575 17248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff8e584259fb4d2cbe80666d006c7745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:06.219712 17524 tablet_service.cc:1467] Processing CreateTablet for tablet ff8e584259fb4d2cbe80666d006c7745 (DEFAULT_TABLE table=test-table [id=d2b1708104e64200a76e248b95883276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:06.219877 16996 tablet_service.cc:1467] Processing CreateTablet for tablet ff8e584259fb4d2cbe80666d006c7745 (DEFAULT_TABLE table=test-table [id=d2b1708104e64200a76e248b95883276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:52:06.220090 17524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff8e584259fb4d2cbe80666d006c7745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:06.220183 16996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff8e584259fb4d2cbe80666d006c7745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:06.222363 17695 tablet_bootstrap.cc:492] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Bootstrap starting.
I20260811 18:52:06.223110 17695 tablet_bootstrap.cc:654] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:06.223418 17695 log.cc:824] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:06.223834 17699 tablet_bootstrap.cc:492] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: Bootstrap starting.
I20260811 18:52:06.224115 17696 tablet_bootstrap.cc:492] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Bootstrap starting.
I20260811 18:52:06.224311 17695 tablet_bootstrap.cc:492] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: No bootstrap required, opened a new log
I20260811 18:52:06.224386 17695 ts_tablet_manager.cc:1397] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 18:52:06.224732 17696 tablet_bootstrap.cc:654] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:06.225001 17696 log.cc:824] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:06.225834 17696 tablet_bootstrap.cc:492] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: No bootstrap required, opened a new log
I20260811 18:52:06.225912 17696 ts_tablet_manager.cc:1397] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:52:06.226063 17699 tablet_bootstrap.cc:654] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:06.226341 17699 log.cc:824] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:06.227049 17699 tablet_bootstrap.cc:492] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: No bootstrap required, opened a new log
I20260811 18:52:06.227124 17699 ts_tablet_manager.cc:1397] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:52:06.227048 17695 raft_consensus.cc:348] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.227238 17695 raft_consensus.cc:374] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:06.227285 17695 raft_consensus.cc:729] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03adb4ebd0f247dcb07c1343cef5029c, State: Initialized, Role: FOLLOWER
I20260811 18:52:06.227414 17695 consensus_queue.cc:260] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.227761 17695 ts_tablet_manager.cc:1428] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 18:52:06.227773 17407 heartbeater.cc:503] Master 127.9.215.62:44567 was elected leader, sending a full tablet report...
I20260811 18:52:06.228487 17696 raft_consensus.cc:348] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.228650 17696 raft_consensus.cc:374] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:06.228693 17696 raft_consensus.cc:729] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb413e7858f14cc5882ec54287977df5, State: Initialized, Role: FOLLOWER
I20260811 18:52:06.228797 17696 consensus_queue.cc:260] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.229061 17696 ts_tablet_manager.cc:1428] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:52:06.229126 17666 heartbeater.cc:503] Master 127.9.215.62:44567 was elected leader, sending a full tablet report...
I20260811 18:52:06.229912 17699 raft_consensus.cc:348] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.230096 17699 raft_consensus.cc:374] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:06.230151 17699 raft_consensus.cc:729] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23eb5ca11e1747a28ea568e769512dea, State: Initialized, Role: FOLLOWER
I20260811 18:52:06.230294 17699 consensus_queue.cc:260] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.230556 17699 ts_tablet_manager.cc:1428] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 18:52:06.230654 17126 heartbeater.cc:503] Master 127.9.215.62:44567 was elected leader, sending a full tablet report...
W20260811 18:52:06.316489 17409 tablet.cc:2369] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:06.399878 17712 raft_consensus.cc:482] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:52:06.400023 17712 raft_consensus.cc:504] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.400408 17712 leader_election.cc:288] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03adb4ebd0f247dcb07c1343cef5029c (127.9.215.2:33253), cb413e7858f14cc5882ec54287977df5 (127.9.215.3:36787)
I20260811 18:52:06.403332 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.400640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:50921 (local address 127.9.215.3:36787)
0811 18:52:06.400728 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.400730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.400853 (+ 123us) server_negotiation.cc:407] Connection header received
0811 18:52:06.400887 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.400888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.400897 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.400959 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:06.401183 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.401630 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.402612 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.402978 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.403040 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.403050 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.403218 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.403262 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:52:06.403362 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.400640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.215.1:43057 (local address 127.9.215.2:33253)
0811 18:52:06.400713 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.400715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.400722 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:52:06.400750 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.400751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.400762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.400815 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:52:06.401033 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.401593 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.402608 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.402973 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.403032 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.403045 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.403249 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.403296 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:52:06.403362 17133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.400569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.2:33253 (local address 127.9.215.1:43057)
0811 18:52:06.400628 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.400641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.400679 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.400852 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.400853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.400855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:06.400936 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.400938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.401688 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.401689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.402485 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.402487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.403099 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.403100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.403178 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.403181 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.403228 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.403272 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 18:52:06.403365 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.400581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.3:36787 (local address 127.9.215.1:50921)
0811 18:52:06.400785 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.400793 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:06.400842 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:06.401006 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:06.401007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:06.401010 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:52:06.401102 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.401104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.401711 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.401712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.402509 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:06.402511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.403061 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.403062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:06.403143 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.403146 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:52:06.403193 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:52:06.403237 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
I20260811 18:52:06.403823 17290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff8e584259fb4d2cbe80666d006c7745" candidate_uuid: "23eb5ca11e1747a28ea568e769512dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03adb4ebd0f247dcb07c1343cef5029c" is_pre_election: true
I20260811 18:52:06.403982 17290 raft_consensus.cc:2454] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23eb5ca11e1747a28ea568e769512dea in term 0.
I20260811 18:52:06.403951 17560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff8e584259fb4d2cbe80666d006c7745" candidate_uuid: "23eb5ca11e1747a28ea568e769512dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb413e7858f14cc5882ec54287977df5" is_pre_election: true
I20260811 18:52:06.404109 17560 raft_consensus.cc:2454] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23eb5ca11e1747a28ea568e769512dea in term 0.
I20260811 18:52:06.404270 16911 leader_election.cc:302] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03adb4ebd0f247dcb07c1343cef5029c, 23eb5ca11e1747a28ea568e769512dea; no voters:
I20260811 18:52:06.404402 17712 raft_consensus.cc:2781] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:52:06.404453 17712 raft_consensus.cc:482] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:52:06.404484 17712 raft_consensus.cc:3037] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:06.405269 17712 raft_consensus.cc:504] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.405421 17712 leader_election.cc:288] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [CANDIDATE]: Term 1 election: Requested vote from peers 03adb4ebd0f247dcb07c1343cef5029c (127.9.215.2:33253), cb413e7858f14cc5882ec54287977df5 (127.9.215.3:36787)
I20260811 18:52:06.405613 17560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff8e584259fb4d2cbe80666d006c7745" candidate_uuid: "23eb5ca11e1747a28ea568e769512dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb413e7858f14cc5882ec54287977df5"
I20260811 18:52:06.405620 17290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff8e584259fb4d2cbe80666d006c7745" candidate_uuid: "23eb5ca11e1747a28ea568e769512dea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03adb4ebd0f247dcb07c1343cef5029c"
I20260811 18:52:06.405711 17290 raft_consensus.cc:3037] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:06.405711 17560 raft_consensus.cc:3037] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:06.406713 17560 raft_consensus.cc:2454] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23eb5ca11e1747a28ea568e769512dea in term 1.
I20260811 18:52:06.406713 17290 raft_consensus.cc:2454] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23eb5ca11e1747a28ea568e769512dea in term 1.
I20260811 18:52:06.406906 16906 leader_election.cc:302] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23eb5ca11e1747a28ea568e769512dea, cb413e7858f14cc5882ec54287977df5; no voters:
I20260811 18:52:06.407013 17712 raft_consensus.cc:2781] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:06.407141 17712 raft_consensus.cc:686] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [term 1 LEADER]: Becoming Leader. State: Replica: 23eb5ca11e1747a28ea568e769512dea, State: Running, Role: LEADER
I20260811 18:52:06.407261 17712 consensus_queue.cc:237] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } }
I20260811 18:52:06.408281 16752 catalog_manager.cc:5495] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea reported cstate change: term changed from 0 to 1, leader changed from <none> to 23eb5ca11e1747a28ea568e769512dea (127.9.215.1). New cstate: current_term: 1 leader_uuid: "23eb5ca11e1747a28ea568e769512dea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23eb5ca11e1747a28ea568e769512dea" member_type: VOTER last_known_addr { host: "127.9.215.1" port: 33933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 } health_report { overall_health: UNKNOWN } } }
W20260811 18:52:06.429229 17132 tablet.cc:2369] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:52:06.446564 17670 tablet.cc:2369] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:52:06.449471 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.447115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.9.215.1:33933)
0811 18:52:06.447171 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:06.447174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:06.447184 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:06.447232 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:06.447233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:06.447256 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:06.447322 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:06.447477 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.447951 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.448720 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:06.448913 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:06.448969 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:06.449122 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:06.449238 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:06.449317 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:06.449340 (+ 23us) server_negotiation.cc:299] Negotiation successful
0811 18:52:06.449363 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 18:52:06.451411 17560 raft_consensus.cc:1261] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 1 FOLLOWER]: Refusing update from remote peer 23eb5ca11e1747a28ea568e769512dea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:06.451404 17290 raft_consensus.cc:1261] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 1 FOLLOWER]: Refusing update from remote peer 23eb5ca11e1747a28ea568e769512dea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:06.451836 17714 consensus_queue.cc:1035] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb413e7858f14cc5882ec54287977df5" member_type: VOTER last_known_addr { host: "127.9.215.3" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:06.451987 17712 consensus_queue.cc:1035] T ff8e584259fb4d2cbe80666d006c7745 P 23eb5ca11e1747a28ea568e769512dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "03adb4ebd0f247dcb07c1343cef5029c" member_type: VOTER last_known_addr { host: "127.9.215.2" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:06.454427 17719 mvcc.cc:205] Tried to move back new op lower bound from 7317398841142116352 to 7317398840964968448. Current Snapshot: MvccSnapshot[applied={T|T < 7317398841142116352}]
I20260811 18:52:06.457636 16752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:06.457772 16752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:52:06.458627 16752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae31472f05c64dc1b20a6ccea7dd4c7f: Sending DeleteTablet for 3 replicas of tablet ff8e584259fb4d2cbe80666d006c7745
I20260811 18:52:06.459046 16996 tablet_service.cc:1514] Processing DeleteTablet for tablet ff8e584259fb4d2cbe80666d006c7745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:06 UTC) from {username='slave'} at 127.0.0.1:35018
I20260811 18:52:06.459041 17248 tablet_service.cc:1514] Processing DeleteTablet for tablet ff8e584259fb4d2cbe80666d006c7745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:06 UTC) from {username='slave'} at 127.0.0.1:33018
I20260811 18:52:06.459255 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16878
I20260811 18:52:06.459347 17524 tablet_service.cc:1514] Processing DeleteTablet for tablet ff8e584259fb4d2cbe80666d006c7745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:52:06 UTC) from {username='slave'} at 127.0.0.1:46412
I20260811 18:52:06.467566 17736 tablet_replica.cc:330] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: stopping tablet replica
I20260811 18:52:06.467764 17736 raft_consensus.cc:2229] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:06.467888 17736 raft_consensus.cc:2258] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:06.473266 17736 ts_tablet_manager.cc:1905] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:06.474822 17738 tablet_replica.cc:330] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: stopping tablet replica
I20260811 18:52:06.474961 17738 raft_consensus.cc:2229] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:52:06.475069 17738 raft_consensus.cc:2258] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:52:06.475236 17736 ts_tablet_manager.cc:1918] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:06.475303 17736 log.cc:1196] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-1/wal/wals/ff8e584259fb4d2cbe80666d006c7745
I20260811 18:52:06.475571 17736 ts_tablet_manager.cc:1939] T ff8e584259fb4d2cbe80666d006c7745 P 03adb4ebd0f247dcb07c1343cef5029c: Deleting consensus metadata
I20260811 18:52:06.476197 16726 catalog_manager.cc:4841] TS 03adb4ebd0f247dcb07c1343cef5029c (127.9.215.2:33253): tablet ff8e584259fb4d2cbe80666d006c7745 (table test-table [id=d2b1708104e64200a76e248b95883276]) successfully deleted
I20260811 18:52:06.476455 17738 ts_tablet_manager.cc:1905] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:52:06.477998 17738 ts_tablet_manager.cc:1918] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:52:06.478099 17738 log.cc:1196] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Deleting WAL directory at /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474068101683-10076-0/minicluster-data/ts-2/wal/wals/ff8e584259fb4d2cbe80666d006c7745
I20260811 18:52:06.478325 17738 ts_tablet_manager.cc:1939] T ff8e584259fb4d2cbe80666d006c7745 P cb413e7858f14cc5882ec54287977df5: Deleting consensus metadata
I20260811 18:52:06.478811 16720 catalog_manager.cc:4841] TS cb413e7858f14cc5882ec54287977df5 (127.9.215.3:36787): tablet ff8e584259fb4d2cbe80666d006c7745 (table test-table [id=d2b1708104e64200a76e248b95883276]) successfully deleted
W20260811 18:52:06.480845 16720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.215.1:33933 (error 108)
I20260811 18:52:06.481384 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:06.481228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.1:33933 (local address 127.0.0.1:35034)
0811 18:52:06.481285 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:06.481327 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.1:33933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 18:52:06.481448 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17134
W20260811 18:52:06.481534 16720 catalog_manager.cc:4584] TS 23eb5ca11e1747a28ea568e769512dea (127.9.215.1:33933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff8e584259fb4d2cbe80666d006c7745: Network error: Client connection negotiation failed: client connection to 127.9.215.1:33933: connect: Connection refused (error 111)
I20260811 18:52:06.505991 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17411
I20260811 18:52:06.518640 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 16686
2026-08-11T18:52:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (948 ms)
[----------] 6 tests from EncryptionPolicyTest (10144 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:06 dist-test-slave-pww0 krb5kdc[17743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:06 dist-test-slave-pww0 krb5kdc[17743](info): set up 2 sockets
Aug 11 18:52:06 dist-test-slave-pww0 krb5kdc[17743](info): commencing operation
krb5kdc: starting...
W20260811 18:52:07.750115 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:07 dist-test-slave-pww0 krb5kdc[17743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:07.883014 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:07.970161 17765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:07.970413 17765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:07.970448 17765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:07.971858 17765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:07.971915 17765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:07.971935 17765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:07.971951 17765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:07.971966 17765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:07.973690 17765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:07.974000 17765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:07.974239 17765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:07.979691 17780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:07.979703 17779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:07.979741 17765 server_base.cc:1034] running on GCE node
W20260811 18:52:07.979734 17784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:07.980305 17765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:07.980628 17765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:07.981788 17765 hybrid_clock.cc:648] HybridClock initialized: now 1786474327981780 us; error 50 us; skew 500 ppm
Aug 11 18:52:07 dist-test-slave-pww0 krb5kdc[17743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474327, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:07.984606 17765 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:07.985062 17765 webserver.cc:460] Webserver started at http://127.0.0.1:33027/ using document root <none> and password file <none>
I20260811 18:52:07.985275 17765 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:07.985317 17765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:07.985417 17765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:07.986479 17765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "79919900aab141ad95a30c078f856d67"
format_stamp: "Formatted at 2026-08-11 18:52:07 on dist-test-slave-pww0"
server_key: "540bfab7692c048c54070d70d52ff1b2"
server_key_iv: "461197c1129013cbbe7b6ad317af160c"
server_key_version: "encryptionkey@0"
I20260811 18:52:07.986801 17765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "79919900aab141ad95a30c078f856d67"
format_stamp: "Formatted at 2026-08-11 18:52:07 on dist-test-slave-pww0"
server_key: "540bfab7692c048c54070d70d52ff1b2"
server_key_iv: "461197c1129013cbbe7b6ad317af160c"
server_key_version: "encryptionkey@0"
I20260811 18:52:07.988224 17765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:07.989044 17801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:07.989225 17765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:07.989323 17765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "79919900aab141ad95a30c078f856d67"
format_stamp: "Formatted at 2026-08-11 18:52:07 on dist-test-slave-pww0"
server_key: "540bfab7692c048c54070d70d52ff1b2"
server_key_iv: "461197c1129013cbbe7b6ad317af160c"
server_key_version: "encryptionkey@0"
I20260811 18:52:07.989387 17765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:08.008322 17765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:08.010706 17765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:08.010841 17765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:08.015327 17765 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39429
I20260811 18:52:08.015383 17914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39429 every 8 connection(s)
I20260811 18:52:08.015811 17765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:08.017045 17915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:08.019658 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 17765
I20260811 18:52:08.019816 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:08.020010 10076 external_mini_cluster.cc:1428] Setting key 7e21d09d43062ea67e2d275aff05db98
I20260811 18:52:08.020053 17915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67: Bootstrap starting.
I20260811 18:52:08.021021 17915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:08.021351 17915 log.cc:824] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:08.022123 17915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67: No bootstrap required, opened a new log
I20260811 18:52:08.024366 17915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79919900aab141ad95a30c078f856d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260811 18:52:08.024542 17915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:08.024575 17915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79919900aab141ad95a30c078f856d67, State: Initialized, Role: FOLLOWER
I20260811 18:52:08.024749 17915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79919900aab141ad95a30c078f856d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260811 18:52:08.024858 17915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:08.024895 17915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:08.024948 17915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:08.025832 17915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79919900aab141ad95a30c078f856d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260811 18:52:08.025982 17915 leader_election.cc:302] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79919900aab141ad95a30c078f856d67; no voters:
I20260811 18:52:08.026307 17915 leader_election.cc:288] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:08.026355 17924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:08.026546 17924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [term 1 LEADER]: Becoming Leader. State: Replica: 79919900aab141ad95a30c078f856d67, State: Running, Role: LEADER
I20260811 18:52:08.026647 17915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:08.026804 17924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79919900aab141ad95a30c078f856d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
Aug 11 18:52:08 dist-test-slave-pww0 krb5kdc[17743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:52:08.027269 17925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79919900aab141ad95a30c078f856d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79919900aab141ad95a30c078f856d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } } }
I20260811 18:52:08.027343 17925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:08.027536 17926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79919900aab141ad95a30c078f856d67. Latest consensus state: current_term: 1 leader_uuid: "79919900aab141ad95a30c078f856d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79919900aab141ad95a30c078f856d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } } }
I20260811 18:52:08.027629 17926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67 [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:08.029265 17948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:08.029572 17948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:08.030385 17930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:08.030568 17930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:08.032155 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:08.020636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55347 (local address 127.0.0.1:39429)
0811 18:52:08.020925 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:08.020934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:08.020967 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:52:08.021554 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:08.021562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:08.021624 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:08.021771 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:08.022462 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:08.023386 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:08.024176 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:08.024633 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:08.024751 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:08.028019 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:08.028024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:08.028024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:08.028037 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:08.030253 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:08.030793 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:08.030797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:08.030797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:08.030909 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:08.031178 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:08.031179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:08.031179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:08.031519 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:08.031668 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:08.031819 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 18:52:08.031926 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260811 18:52:08.033140 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:08.040098 17930 catalog_manager.cc:1347] Generated new cluster ID: e2f1874e607f4849aa832a1a5b14470e
I20260811 18:52:08.040177 17930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:08.040928 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:08.034527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56533 (local address 127.0.0.1:39429)
0811 18:52:08.034565 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:08.034567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:08.034675 (+ 108us) server_negotiation.cc:407] Connection header received
0811 18:52:08.034773 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:08.034774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:08.034784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:08.034829 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:08.035332 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:08.035887 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:08.036584 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:08.036820 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:08.036877 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:08.038100 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:08.038101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:08.038102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:08.038110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:08.039422 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:08.039830 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:08.039831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:08.039831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:08.039885 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:08.040132 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:08.040133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:08.040133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:08.040280 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:08.040396 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:08.040769 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 18:52:08.040833 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 18:52:08.054890 17930 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:08.055526 17930 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:08.062466 17930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79919900aab141ad95a30c078f856d67: Generated new TSK 0
I20260811 18:52:08.062712 17930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:08.073861 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 17765
W20260811 18:52:08.091421 17740 connection.cc:535] client connection to 127.0.0.1:39429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39429): Connection reset by peer (error 104)
2026-08-11T18:52:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:08 dist-test-slave-pww0 krb5kdc[17968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:08 dist-test-slave-pww0 krb5kdc[17968](info): set up 2 sockets
Aug 11 18:52:08 dist-test-slave-pww0 krb5kdc[17968](info): commencing operation
krb5kdc: starting...
W20260811 18:52:09.311900 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:09 dist-test-slave-pww0 krb5kdc[17968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:09.443356 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:09.529363 18005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:09.529598 18005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:09.529633 18005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:09.530901 18005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:09.530956 18005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:09.530974 18005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:09.530992 18005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:09.531008 18005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:09.532717 18005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35923
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:09.533018 18005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:09.533253 18005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:09.538645 18019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:09.538663 18016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:09.538653 18014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:09.538691 18005 server_base.cc:1034] running on GCE node
I20260811 18:52:09.539182 18005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:09.539566 18005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:09.540755 18005 hybrid_clock.cc:648] HybridClock initialized: now 1786474329540739 us; error 33 us; skew 500 ppm
Aug 11 18:52:09 dist-test-slave-pww0 krb5kdc[17968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:09.547686 18005 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:09.548233 18005 webserver.cc:460] Webserver started at http://127.0.0.1:43025/ using document root <none> and password file <none>
I20260811 18:52:09.548471 18005 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:09.548523 18005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:09.548663 18005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:09.549832 18005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "a3547c6d144d48079f44bbcf44bac761"
format_stamp: "Formatted at 2026-08-11 18:52:09 on dist-test-slave-pww0"
server_key: "d29354da45591c1170b12f26d4f96023"
server_key_iv: "855911d890c7a1a34ec213b1cbea99a4"
server_key_version: "encryptionkey@0"
I20260811 18:52:09.550184 18005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "a3547c6d144d48079f44bbcf44bac761"
format_stamp: "Formatted at 2026-08-11 18:52:09 on dist-test-slave-pww0"
server_key: "d29354da45591c1170b12f26d4f96023"
server_key_iv: "855911d890c7a1a34ec213b1cbea99a4"
server_key_version: "encryptionkey@0"
I20260811 18:52:09.551625 18005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 18:52:09.552508 18029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:09.552703 18005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:09.552801 18005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "a3547c6d144d48079f44bbcf44bac761"
format_stamp: "Formatted at 2026-08-11 18:52:09 on dist-test-slave-pww0"
server_key: "d29354da45591c1170b12f26d4f96023"
server_key_iv: "855911d890c7a1a34ec213b1cbea99a4"
server_key_version: "encryptionkey@0"
I20260811 18:52:09.552870 18005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:09.583849 18005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:09.586140 18005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:09.586301 18005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:09.590627 18005 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35923
I20260811 18:52:09.590682 18115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35923 every 8 connection(s)
I20260811 18:52:09.591038 18005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:09.592020 18119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:09.594404 18119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761: Bootstrap starting.
I20260811 18:52:09.595022 18119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:09.595294 18119 log.cc:824] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:09.596069 18119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761: No bootstrap required, opened a new log
I20260811 18:52:09.597810 18119 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3547c6d144d48079f44bbcf44bac761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35923 } }
I20260811 18:52:09.597965 18119 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:09.597987 18119 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3547c6d144d48079f44bbcf44bac761, State: Initialized, Role: FOLLOWER
I20260811 18:52:09.598129 18119 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3547c6d144d48079f44bbcf44bac761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35923 } }
I20260811 18:52:09.598196 18119 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:09.598228 18119 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:09.598275 18119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:09.598868 18119 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3547c6d144d48079f44bbcf44bac761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35923 } }
I20260811 18:52:09.598973 18119 leader_election.cc:302] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3547c6d144d48079f44bbcf44bac761; no voters:
I20260811 18:52:09.599151 18119 leader_election.cc:288] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:09.599202 18122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:09.599390 18122 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [term 1 LEADER]: Becoming Leader. State: Replica: a3547c6d144d48079f44bbcf44bac761, State: Running, Role: LEADER
I20260811 18:52:09.599413 18119 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:09.599597 18122 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3547c6d144d48079f44bbcf44bac761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35923 } }
I20260811 18:52:09.600111 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18005
I20260811 18:52:09.600230 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:09.600198 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3547c6d144d48079f44bbcf44bac761. Latest consensus state: current_term: 1 leader_uuid: "a3547c6d144d48079f44bbcf44bac761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3547c6d144d48079f44bbcf44bac761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35923 } } }
I20260811 18:52:09.600275 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:09.600435 10076 external_mini_cluster.cc:1428] Setting key f8b97ef06f73363b5a9b050cfed34a09
I20260811 18:52:09.600680 18142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:09.600931 18124 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3547c6d144d48079f44bbcf44bac761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3547c6d144d48079f44bbcf44bac761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35923 } } }
I20260811 18:52:09.601121 18124 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:09.601336 18142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:09.608448 18142 catalog_manager.cc:1347] Generated new cluster ID: ebe4e2e8b8a94c34a6d8ed8496005ba5
I20260811 18:52:09.608536 18142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:09.609357 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:09.601139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36415 (local address 127.0.0.1:35923)
0811 18:52:09.601309 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:09.601319 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:09.603776 (+ 2457us) server_negotiation.cc:407] Connection header received
0811 18:52:09.603924 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:09.603931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:09.603983 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:09.604166 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:09.604527 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:09.605553 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:09.606367 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:09.607398 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:09.607555 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:09.608522 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:09.608527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:09.608527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:09.608770 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:09.608812 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:09.608828 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:09.609052 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:09.609093 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 18:52:09.609124 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260811 18:52:09.617822 18142 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:09.618363 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:09.618618 18142 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:09.622887 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:09.619876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51561 (local address 127.0.0.1:35923)
0811 18:52:09.619920 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:09.619922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:09.620032 (+ 110us) server_negotiation.cc:407] Connection header received
0811 18:52:09.620099 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:09.620100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:09.620111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:09.620154 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:09.620303 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:09.620864 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:09.621625 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:09.621861 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:09.621918 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:09.622632 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:09.622633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:09.622633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:09.622640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:09.622656 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:09.622661 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:09.622710 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:09.622780 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 18:52:09.622795 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:52:09.630359 18142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3547c6d144d48079f44bbcf44bac761: Generated new TSK 0
I20260811 18:52:09.630622 18142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:09.634434 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18005
W20260811 18:52:09.675917 17965 connection.cc:535] client connection to 127.0.0.1:35923 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35923): Connection reset by peer (error 104)
2026-08-11T18:52:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:09 dist-test-slave-pww0 krb5kdc[18180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:09 dist-test-slave-pww0 krb5kdc[18180](info): set up 2 sockets
Aug 11 18:52:09 dist-test-slave-pww0 krb5kdc[18180](info): commencing operation
krb5kdc: starting...
W20260811 18:52:10.887214 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:10 dist-test-slave-pww0 krb5kdc[18180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:11.027984 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:11.117128 18209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:11.117380 18209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:11.117408 18209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:11.118681 18209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:11.118738 18209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:11.118752 18209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:11.118763 18209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:11.118774 18209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:11.120414 18209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:11.120687 18209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:11.120905 18209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:11.125989 18216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:11.126117 18215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:11.125984 18218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:11.126353 18209 server_base.cc:1034] running on GCE node
I20260811 18:52:11.126742 18209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:11.127125 18209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:11.128288 18209 hybrid_clock.cc:648] HybridClock initialized: now 1786474331128272 us; error 30 us; skew 500 ppm
Aug 11 18:52:11 dist-test-slave-pww0 krb5kdc[18180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474331, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:11.131913 18209 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:11.132550 18209 webserver.cc:460] Webserver started at http://127.0.0.1:38381/ using document root <none> and password file <none>
I20260811 18:52:11.132840 18209 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:11.132906 18209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:11.133062 18209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:11.134517 18209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "9120cb512b684b85a8fda1dca698d92d"
format_stamp: "Formatted at 2026-08-11 18:52:11 on dist-test-slave-pww0"
server_key: "ff9fdb10d51c53e43963a2eb8c625d57"
server_key_iv: "d58fbddecbc1f390a25357cd38dc44ec"
server_key_version: "encryptionkey@0"
I20260811 18:52:11.134878 18209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "9120cb512b684b85a8fda1dca698d92d"
format_stamp: "Formatted at 2026-08-11 18:52:11 on dist-test-slave-pww0"
server_key: "ff9fdb10d51c53e43963a2eb8c625d57"
server_key_iv: "d58fbddecbc1f390a25357cd38dc44ec"
server_key_version: "encryptionkey@0"
I20260811 18:52:11.136384 18209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:52:11.137247 18230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:11.137403 18209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:11.137507 18209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "9120cb512b684b85a8fda1dca698d92d"
format_stamp: "Formatted at 2026-08-11 18:52:11 on dist-test-slave-pww0"
server_key: "ff9fdb10d51c53e43963a2eb8c625d57"
server_key_iv: "d58fbddecbc1f390a25357cd38dc44ec"
server_key_version: "encryptionkey@0"
I20260811 18:52:11.137584 18209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:11.141608 18209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:11.143923 18209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:11.144084 18209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:11.148993 18209 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42841
I20260811 18:52:11.149093 18318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42841 every 8 connection(s)
I20260811 18:52:11.149436 18209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:11.150388 18320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:11.153014 18320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d: Bootstrap starting.
I20260811 18:52:11.153682 18320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:11.153963 18320 log.cc:824] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:11.154323 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18209
I20260811 18:52:11.154439 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:11.154644 10076 external_mini_cluster.cc:1428] Setting key d5b5f13aff3679ce134988c1a648777d
I20260811 18:52:11.154799 18320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d: No bootstrap required, opened a new log
I20260811 18:52:11.157162 18320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120cb512b684b85a8fda1dca698d92d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42841 } }
I20260811 18:52:11.157380 18320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:11.157414 18320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9120cb512b684b85a8fda1dca698d92d, State: Initialized, Role: FOLLOWER
I20260811 18:52:11.157851 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:11.155246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43935 (local address 127.0.0.1:42841)
0811 18:52:11.155529 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:11.155542 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:11.155597 (+ 55us) server_negotiation.cc:407] Connection header received
0811 18:52:11.156272 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:11.156283 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:11.156336 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:11.156510 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:11.157196 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:11.157199 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:11.157200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:11.157205 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:11.157253 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:11.157269 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:11.157348 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:11.157474 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 18:52:11.157640 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":115,"threads_started":1}
I20260811 18:52:11.157574 18320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120cb512b684b85a8fda1dca698d92d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42841 } }
I20260811 18:52:11.157991 18320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:11.158041 18320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:11.158113 18320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:11.158866 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:11.158876 18320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120cb512b684b85a8fda1dca698d92d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42841 } }
I20260811 18:52:11.158987 18320 leader_election.cc:302] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9120cb512b684b85a8fda1dca698d92d; no voters:
I20260811 18:52:11.159173 18320 leader_election.cc:288] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:11.159214 18330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:11.159340 18330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [term 1 LEADER]: Becoming Leader. State: Replica: 9120cb512b684b85a8fda1dca698d92d, State: Running, Role: LEADER
I20260811 18:52:11.159497 18320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:11.159516 18330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120cb512b684b85a8fda1dca698d92d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42841 } }
I20260811 18:52:11.159917 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9120cb512b684b85a8fda1dca698d92d. Latest consensus state: current_term: 1 leader_uuid: "9120cb512b684b85a8fda1dca698d92d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120cb512b684b85a8fda1dca698d92d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42841 } } }
I20260811 18:52:11.160009 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:11.160099 18331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9120cb512b684b85a8fda1dca698d92d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9120cb512b684b85a8fda1dca698d92d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42841 } } }
I20260811 18:52:11.160205 18331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:11.161564 18367 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:11.161630 18367 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:11.161674 18348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:11.161860 18348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:11.162302 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:11.160984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37107 (local address 127.0.0.1:42841)
0811 18:52:11.161041 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:11.161042 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:11.161239 (+ 197us) server_negotiation.cc:407] Connection header received
0811 18:52:11.161337 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:11.161338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:11.161346 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:11.161400 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:11.162009 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:11.162011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:11.162011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:11.162015 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:11.162033 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:11.162039 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:11.162104 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:11.162227 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 18:52:11.162241 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:52:11.163568 18348 catalog_manager.cc:1347] Generated new cluster ID: 9d0ee533c81e43468ba397adde070987
I20260811 18:52:11.163630 18348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:11.169827 18348 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:11.170030 18348 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:11.177487 18348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9120cb512b684b85a8fda1dca698d92d: Generated new TSK 0
I20260811 18:52:11.177704 18348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:11.181314 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18209
2026-08-11T18:52:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:11 dist-test-slave-pww0 krb5kdc[18377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:11 dist-test-slave-pww0 krb5kdc[18377](info): set up 2 sockets
Aug 11 18:52:11 dist-test-slave-pww0 krb5kdc[18377](info): commencing operation
krb5kdc: starting...
W20260811 18:52:12.403051 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:12 dist-test-slave-pww0 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:12.535255 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:12.619534 18403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:12.619789 18403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:12.619822 18403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:12.621094 18403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:12.621152 18403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:12.621171 18403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:12.621188 18403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:12.621203 18403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:12.622924 18403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:12.623245 18403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:12.623521 18403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:12.628865 18415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:12.629040 18409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:12.628865 18410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:12.629092 18403 server_base.cc:1034] running on GCE node
I20260811 18:52:12.629343 18403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:12.629690 18403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:12.630860 18403 hybrid_clock.cc:648] HybridClock initialized: now 1786474332630845 us; error 32 us; skew 500 ppm
Aug 11 18:52:12 dist-test-slave-pww0 krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:12.633509 18403 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:12.633985 18403 webserver.cc:460] Webserver started at http://127.0.0.1:38231/ using document root <none> and password file <none>
I20260811 18:52:12.634198 18403 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:12.634241 18403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:12.634342 18403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:12.635403 18403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "28880c73012d479bbceeac4c34313825"
format_stamp: "Formatted at 2026-08-11 18:52:12 on dist-test-slave-pww0"
server_key: "770d8ed1eb1b6c28bdf23d222fa75c7f"
server_key_iv: "a1e8ef5cd60e292b386ebcbd545d3e54"
server_key_version: "encryptionkey@0"
I20260811 18:52:12.635771 18403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "28880c73012d479bbceeac4c34313825"
format_stamp: "Formatted at 2026-08-11 18:52:12 on dist-test-slave-pww0"
server_key: "770d8ed1eb1b6c28bdf23d222fa75c7f"
server_key_iv: "a1e8ef5cd60e292b386ebcbd545d3e54"
server_key_version: "encryptionkey@0"
I20260811 18:52:12.637140 18403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:12.638079 18425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:12.638293 18403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:12.638401 18403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "28880c73012d479bbceeac4c34313825"
format_stamp: "Formatted at 2026-08-11 18:52:12 on dist-test-slave-pww0"
server_key: "770d8ed1eb1b6c28bdf23d222fa75c7f"
server_key_iv: "a1e8ef5cd60e292b386ebcbd545d3e54"
server_key_version: "encryptionkey@0"
I20260811 18:52:12.638479 18403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:12.674875 18403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:12.677090 18403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:12.677230 18403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:12.681272 18403 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44673
I20260811 18:52:12.681348 18527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44673 every 8 connection(s)
I20260811 18:52:12.681689 18403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:12.682617 18528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:12.685055 18528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825: Bootstrap starting.
I20260811 18:52:12.685674 18528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:12.685945 18528 log.cc:824] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:12.686699 18528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825: No bootstrap required, opened a new log
I20260811 18:52:12.688472 18528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28880c73012d479bbceeac4c34313825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260811 18:52:12.688637 18528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:12.688673 18528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28880c73012d479bbceeac4c34313825, State: Initialized, Role: FOLLOWER
I20260811 18:52:12.688827 18528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28880c73012d479bbceeac4c34313825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260811 18:52:12.688913 18528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:12.688947 18528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:12.688999 18528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:12.689592 18528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28880c73012d479bbceeac4c34313825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260811 18:52:12.689702 18528 leader_election.cc:302] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28880c73012d479bbceeac4c34313825; no voters:
I20260811 18:52:12.689884 18528 leader_election.cc:288] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:12.689971 18533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:12.690166 18528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:12.690176 18533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [term 1 LEADER]: Becoming Leader. State: Replica: 28880c73012d479bbceeac4c34313825, State: Running, Role: LEADER
I20260811 18:52:12.690318 18533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28880c73012d479bbceeac4c34313825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260811 18:52:12.690716 18535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28880c73012d479bbceeac4c34313825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28880c73012d479bbceeac4c34313825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } } }
I20260811 18:52:12.690714 18536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28880c73012d479bbceeac4c34313825. Latest consensus state: current_term: 1 leader_uuid: "28880c73012d479bbceeac4c34313825" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28880c73012d479bbceeac4c34313825" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } } }
I20260811 18:52:12.690819 18535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:12.690824 18536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:12.691042 18542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:12.691663 18542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:12.691826 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18403
I20260811 18:52:12.691939 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:12.692164 10076 external_mini_cluster.cc:1428] Setting key 5d27a4fbc131460297d81708058d7655
I20260811 18:52:12.696696 18542 catalog_manager.cc:1347] Generated new cluster ID: 413fd4258be84509bf4523f55071104c
I20260811 18:52:12.696780 18542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 18:52:12 dist-test-slave-pww0 krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:52:12.703856 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:12.692641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45461 (local address 127.0.0.1:44673)
0811 18:52:12.692801 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:12.692811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:12.692841 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:52:12.692943 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:12.692949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:12.692995 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:12.693109 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:12.693942 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:12.694913 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:12.696065 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:12.696689 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:12.696807 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:12.699747 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:12.699751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:12.699752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:12.699766 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:12.701890 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:12.702405 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:12.702409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:12.702410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:12.702539 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:12.702759 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:12.702760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:12.702761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:12.703090 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:12.703255 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:12.703370 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 18:52:12.703557 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260811 18:52:12.704980 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:12.712097 18542 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:12.713068 18542 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:12.714154 18571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:12.706603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55659 (local address 127.0.0.1:44673)
0811 18:52:12.706651 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:12.706653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:12.706867 (+ 214us) server_negotiation.cc:407] Connection header received
0811 18:52:12.706919 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:12.706920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:12.706931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:12.706984 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:12.707752 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:12.708493 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:12.709549 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:12.709876 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:12.709936 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:12.711438 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:12.711439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:12.711439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:12.711447 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:12.713019 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:12.713418 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:12.713419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:12.713420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:12.713479 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:12.713670 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:12.713671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:12.713671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:12.713781 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:12.713881 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:12.713966 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 18:52:12.714058 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:52:12.721980 18542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28880c73012d479bbceeac4c34313825: Generated new TSK 0
I20260811 18:52:12.722183 18542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:12.726022 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18403
2026-08-11T18:52:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:12 dist-test-slave-pww0 krb5kdc[18588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:12 dist-test-slave-pww0 krb5kdc[18588](info): set up 2 sockets
Aug 11 18:52:12 dist-test-slave-pww0 krb5kdc[18588](info): commencing operation
krb5kdc: starting...
W20260811 18:52:13.972154 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:14 dist-test-slave-pww0 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:14.114394 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:14.219205 18613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:14.219451 18613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:14.219514 18613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:14.220932 18613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:14.221002 18613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:14.221016 18613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:14.221028 18613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:14.221040 18613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:14.222846 18613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:14.223187 18613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:14.223438 18613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:14.229267 18629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:14.229223 18633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:14.229245 18627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:14.229683 18613 server_base.cc:1034] running on GCE node
I20260811 18:52:14.229931 18613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:14.230240 18613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:14.231431 18613 hybrid_clock.cc:648] HybridClock initialized: now 1786474334231415 us; error 33 us; skew 500 ppm
Aug 11 18:52:14 dist-test-slave-pww0 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474334, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:14.234424 18613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:14.234968 18613 webserver.cc:460] Webserver started at http://127.0.0.1:36645/ using document root <none> and password file <none>
I20260811 18:52:14.235203 18613 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:14.235251 18613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:14.235375 18613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:14.236570 18613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "3015b485a239405a88005c797da2d378"
format_stamp: "Formatted at 2026-08-11 18:52:14 on dist-test-slave-pww0"
server_key: "53fe8766c50b435909f6abf38b41f68d"
server_key_iv: "c083bd9fca97a73b39f49f31bd5d7e74"
server_key_version: "encryptionkey@0"
I20260811 18:52:14.236917 18613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "3015b485a239405a88005c797da2d378"
format_stamp: "Formatted at 2026-08-11 18:52:14 on dist-test-slave-pww0"
server_key: "53fe8766c50b435909f6abf38b41f68d"
server_key_iv: "c083bd9fca97a73b39f49f31bd5d7e74"
server_key_version: "encryptionkey@0"
I20260811 18:52:14.238426 18613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:14.239383 18643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:14.239629 18613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:14.239748 18613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "3015b485a239405a88005c797da2d378"
format_stamp: "Formatted at 2026-08-11 18:52:14 on dist-test-slave-pww0"
server_key: "53fe8766c50b435909f6abf38b41f68d"
server_key_iv: "c083bd9fca97a73b39f49f31bd5d7e74"
server_key_version: "encryptionkey@0"
I20260811 18:52:14.239823 18613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:14.265666 18613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:14.268115 18613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:14.268280 18613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:14.272811 18613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46001
I20260811 18:52:14.272861 18750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46001 every 8 connection(s)
I20260811 18:52:14.273239 18613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:14.274209 18752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:14.276916 18752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378: Bootstrap starting.
I20260811 18:52:14.277590 18752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:14.277881 18752 log.cc:824] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:14.278793 18752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378: No bootstrap required, opened a new log
I20260811 18:52:14.279275 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18613
I20260811 18:52:14.279397 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:14.279605 10076 external_mini_cluster.cc:1428] Setting key 79d4ad4cef21697323dc81d9a16bdca7
I20260811 18:52:14.280838 18752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3015b485a239405a88005c797da2d378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46001 } }
I20260811 18:52:14.280998 18752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:14.281032 18752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3015b485a239405a88005c797da2d378, State: Initialized, Role: FOLLOWER
I20260811 18:52:14.281208 18752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3015b485a239405a88005c797da2d378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46001 } }
I20260811 18:52:14.281293 18752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:14.281373 18752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:14.281492 18752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:14.282217 18752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3015b485a239405a88005c797da2d378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46001 } }
I20260811 18:52:14.282326 18752 leader_election.cc:302] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3015b485a239405a88005c797da2d378; no voters:
I20260811 18:52:14.282508 18752 leader_election.cc:288] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:14.282554 18764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:14.282716 18764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [term 1 LEADER]: Becoming Leader. State: Replica: 3015b485a239405a88005c797da2d378, State: Running, Role: LEADER
I20260811 18:52:14.282824 18752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:14.282862 18764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3015b485a239405a88005c797da2d378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46001 } }
I20260811 18:52:14.283265 18764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3015b485a239405a88005c797da2d378. Latest consensus state: current_term: 1 leader_uuid: "3015b485a239405a88005c797da2d378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3015b485a239405a88005c797da2d378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46001 } } }
I20260811 18:52:14.283311 18767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3015b485a239405a88005c797da2d378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3015b485a239405a88005c797da2d378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46001 } } }
I20260811 18:52:14.283432 18767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:14.283664 18764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:14.283869 18779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:14.284474 18779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:14.286636 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:14.280196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33637 (local address 127.0.0.1:46001)
0811 18:52:14.280398 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:14.280411 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:14.280470 (+ 59us) server_negotiation.cc:407] Connection header received
0811 18:52:14.280788 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:14.280794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:14.280854 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:14.280977 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:14.281208 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:14.282509 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:14.283626 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:14.284274 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:14.284389 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:14.285832 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:14.285837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:14.285837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:14.286067 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:14.286107 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:14.286123 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:14.286206 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:14.286312 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 18:52:14.286390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":70,"threads_started":1}
I20260811 18:52:14.286660 18779 catalog_manager.cc:1347] Generated new cluster ID: ae20105790d84451ae00c7d0f33864d0
I20260811 18:52:14.286712 18779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:14.287515 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:14.294998 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:14.291923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36379 (local address 127.0.0.1:46001)
0811 18:52:14.291978 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:14.291980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:14.292262 (+ 282us) server_negotiation.cc:407] Connection header received
0811 18:52:14.292316 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:14.292317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:14.292330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:14.292378 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:14.292525 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:14.293046 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:14.293794 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:14.293991 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:14.294047 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:14.294743 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:14.294745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:14.294745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:14.294751 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:14.294768 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:14.294773 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:14.294817 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:14.294878 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 18:52:14.294908 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:52:14.324117 18779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:14.324752 18779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:14.332937 18779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3015b485a239405a88005c797da2d378: Generated new TSK 0
I20260811 18:52:14.333153 18779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:14.343793 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18613
2026-08-11T18:52:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1618 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:14 dist-test-slave-pww0 krb5kdc[18810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:14 dist-test-slave-pww0 krb5kdc[18810](info): set up 2 sockets
Aug 11 18:52:14 dist-test-slave-pww0 krb5kdc[18810](info): commencing operation
krb5kdc: starting...
W20260811 18:52:15.591836 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:15 dist-test-slave-pww0 krb5kdc[18810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:15.726157 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:15.811975 18838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:15.812218 18838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:15.812256 18838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:15.813560 18838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:15.813616 18838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:15.813635 18838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:15.813652 18838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:15.813668 18838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:15.815348 18838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:15.815659 18838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:15.815904 18838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:15.821252 18849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:15.821280 18850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:15.821386 18855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:15.821482 18838 server_base.cc:1034] running on GCE node
I20260811 18:52:15.821735 18838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:15.822064 18838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:15.823264 18838 hybrid_clock.cc:648] HybridClock initialized: now 1786474335823245 us; error 28 us; skew 500 ppm
Aug 11 18:52:15 dist-test-slave-pww0 krb5kdc[18810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474335, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:15.833280 18838 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:15.833809 18838 webserver.cc:460] Webserver started at http://127.0.0.1:35479/ using document root <none> and password file <none>
I20260811 18:52:15.834038 18838 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:15.834105 18838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:15.834242 18838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:15.835410 18838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "4fa9d2dcd961485eaa10b1ced1226d9b"
format_stamp: "Formatted at 2026-08-11 18:52:15 on dist-test-slave-pww0"
server_key: "80c039b0fe324f139ac0de6b082ba1bc"
server_key_iv: "353893a231341b46cfa6584c3a83da79"
server_key_version: "encryptionkey@0"
I20260811 18:52:15.835824 18838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "4fa9d2dcd961485eaa10b1ced1226d9b"
format_stamp: "Formatted at 2026-08-11 18:52:15 on dist-test-slave-pww0"
server_key: "80c039b0fe324f139ac0de6b082ba1bc"
server_key_iv: "353893a231341b46cfa6584c3a83da79"
server_key_version: "encryptionkey@0"
I20260811 18:52:15.837248 18838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:15.838137 18870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:15.838415 18838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:15.838524 18838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "4fa9d2dcd961485eaa10b1ced1226d9b"
format_stamp: "Formatted at 2026-08-11 18:52:15 on dist-test-slave-pww0"
server_key: "80c039b0fe324f139ac0de6b082ba1bc"
server_key_iv: "353893a231341b46cfa6584c3a83da79"
server_key_version: "encryptionkey@0"
I20260811 18:52:15.838598 18838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:15.842352 18838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:15.844887 18838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:15.845042 18838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:15.849524 18838 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36375
I20260811 18:52:15.849570 18969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36375 every 8 connection(s)
I20260811 18:52:15.849925 18838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:15.851060 18970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:15.852273 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 18838
I20260811 18:52:15.852399 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:15.852607 10076 external_mini_cluster.cc:1428] Setting key aaea139ad4186539b0eaf44122018b96
I20260811 18:52:15.854115 18970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b: Bootstrap starting.
I20260811 18:52:15.854869 18970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:15.855187 18970 log.cc:824] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:15.855701 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:15.853217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56293 (local address 127.0.0.1:36375)
0811 18:52:15.853383 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:15.853392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:15.853420 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:52:15.853927 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:15.853937 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:15.853968 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:15.854201 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:15.855013 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:15.855021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:15.855022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:15.855031 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:15.855178 (+ 147us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:15.855202 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:15.855336 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:15.855383 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 18:52:15.855431 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260811 18:52:15.856056 18970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b: No bootstrap required, opened a new log
I20260811 18:52:15.857218 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:15.858731 18970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260811 18:52:15.858909 18970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:15.858944 18970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa9d2dcd961485eaa10b1ced1226d9b, State: Initialized, Role: FOLLOWER
I20260811 18:52:15.859102 18970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260811 18:52:15.859213 18970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:15.859251 18970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:15.859335 18970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:15.860092 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:15.858994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34289 (local address 127.0.0.1:36375)
0811 18:52:15.859037 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:15.859039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:15.859145 (+ 106us) server_negotiation.cc:407] Connection header received
0811 18:52:15.859186 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:15.859187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:15.859192 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:15.859259 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:15.859886 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:15.859887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:15.859888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:15.859892 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:15.859910 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:15.859914 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:15.859977 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:15.860017 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 18:52:15.860030 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:15.860109 18970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260811 18:52:15.860235 18970 leader_election.cc:302] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fa9d2dcd961485eaa10b1ced1226d9b; no voters:
I20260811 18:52:15.860399 18970 leader_election.cc:288] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:15.860433 18994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:15.860584 18994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [term 1 LEADER]: Becoming Leader. State: Replica: 4fa9d2dcd961485eaa10b1ced1226d9b, State: Running, Role: LEADER
I20260811 18:52:15.860680 18970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:15.860697 18994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } }
I20260811 18:52:15.861050 18994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fa9d2dcd961485eaa10b1ced1226d9b. Latest consensus state: current_term: 1 leader_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } } }
I20260811 18:52:15.861076 18997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa9d2dcd961485eaa10b1ced1226d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36375 } } }
I20260811 18:52:15.861141 18994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:15.861196 18997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:15.861406 19004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:15.861974 19004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:15.864817 19004 catalog_manager.cc:1347] Generated new cluster ID: 29ca263ef9064c5ba7068fb6b1ac213c
I20260811 18:52:15.864894 19004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:15.877379 19004 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:15.877601 19004 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:15.881120 19004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fa9d2dcd961485eaa10b1ced1226d9b: Generated new TSK 0
I20260811 18:52:15.881300 19004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:15.888345 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 18838
2026-08-11T18:52:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:15 dist-test-slave-pww0 krb5kdc[19030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:15 dist-test-slave-pww0 krb5kdc[19030](info): set up 2 sockets
Aug 11 18:52:15 dist-test-slave-pww0 krb5kdc[19030](info): commencing operation
krb5kdc: starting...
W20260811 18:52:17.118868 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:17 dist-test-slave-pww0 krb5kdc[19030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:17.250010 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:17.335904 19051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:17.336134 19051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:17.336167 19051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:17.337411 19051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:17.337466 19051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:17.337484 19051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:17.337500 19051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:17.337517 19051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:17.339139 19051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:17.339418 19051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:17.339673 19051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:17.345186 19071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:17.345297 19051 server_base.cc:1034] running on GCE node
W20260811 18:52:17.345196 19063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:17.345263 19059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:17.345736 19051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:17.346063 19051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:17.347235 19051 hybrid_clock.cc:648] HybridClock initialized: now 1786474337347217 us; error 37 us; skew 500 ppm
Aug 11 18:52:17 dist-test-slave-pww0 krb5kdc[19030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474337, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:17.349903 19051 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:17.350447 19051 webserver.cc:460] Webserver started at http://127.0.0.1:33709/ using document root <none> and password file <none>
I20260811 18:52:17.350672 19051 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:17.350715 19051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:17.350814 19051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:17.351958 19051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b"
format_stamp: "Formatted at 2026-08-11 18:52:17 on dist-test-slave-pww0"
server_key: "b2b0d37b6ddbb741efa89b37d73c1c25"
server_key_iv: "cc8c64aaa00e953d8511cb22ff85ed10"
server_key_version: "encryptionkey@0"
I20260811 18:52:17.352279 19051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b"
format_stamp: "Formatted at 2026-08-11 18:52:17 on dist-test-slave-pww0"
server_key: "b2b0d37b6ddbb741efa89b37d73c1c25"
server_key_iv: "cc8c64aaa00e953d8511cb22ff85ed10"
server_key_version: "encryptionkey@0"
I20260811 18:52:17.353698 19051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:17.354557 19087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:17.354746 19051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:17.354835 19051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b"
format_stamp: "Formatted at 2026-08-11 18:52:17 on dist-test-slave-pww0"
server_key: "b2b0d37b6ddbb741efa89b37d73c1c25"
server_key_iv: "cc8c64aaa00e953d8511cb22ff85ed10"
server_key_version: "encryptionkey@0"
I20260811 18:52:17.354899 19051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:17.373440 19051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:17.375830 19051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:17.375968 19051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:17.379957 19051 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33021
I20260811 18:52:17.379997 19182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33021 every 8 connection(s)
I20260811 18:52:17.380362 19051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:17.381310 19183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:17.383867 19183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b: Bootstrap starting.
I20260811 18:52:17.384526 19183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:17.384811 19183 log.cc:824] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:17.385607 19183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b: No bootstrap required, opened a new log
I20260811 18:52:17.385929 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19051
I20260811 18:52:17.386029 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:17.386212 10076 external_mini_cluster.cc:1428] Setting key 989af95147f19d6bc582b11dfd16360f
I20260811 18:52:17.387955 19183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260811 18:52:17.388106 19183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:17.388139 19183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed45c6e8f054b6592a9a96e05c5ac5b, State: Initialized, Role: FOLLOWER
I20260811 18:52:17.388276 19183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260811 18:52:17.388350 19183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:17.388397 19183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:17.388440 19183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:17.389273 19183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260811 18:52:17.389394 19183 leader_election.cc:302] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ed45c6e8f054b6592a9a96e05c5ac5b; no voters:
I20260811 18:52:17.389576 19183 leader_election.cc:288] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:17.389631 19195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:17.389798 19195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [term 1 LEADER]: Becoming Leader. State: Replica: 1ed45c6e8f054b6592a9a96e05c5ac5b, State: Running, Role: LEADER
I20260811 18:52:17.389837 19183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:17.391088 19195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } }
I20260811 18:52:17.391734 19197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } } }
I20260811 18:52:17.391961 19197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:17.391733 19196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ed45c6e8f054b6592a9a96e05c5ac5b. Latest consensus state: current_term: 1 leader_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ed45c6e8f054b6592a9a96e05c5ac5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33021 } } }
I20260811 18:52:17.392132 19196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:17.392539 19230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:17.392599 19230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:17.392709 19237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:17.392896 19237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 18:52:17 dist-test-slave-pww0 krb5kdc[19030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:52:17.394650 19237 catalog_manager.cc:1347] Generated new cluster ID: b3550b1b72c94ec2b667047beadcf874
I20260811 18:52:17.394702 19237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:17.398924 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:17.386757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:47351 (local address 127.0.0.1:33021)
0811 18:52:17.386985 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:17.386996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:17.387047 (+ 51us) server_negotiation.cc:407] Connection header received
0811 18:52:17.387505 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:17.387516 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:17.387590 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:17.387771 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:17.388425 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:17.390086 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:17.391072 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:17.391678 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:17.391802 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:17.394797 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:17.394801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:17.394802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:17.394816 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:17.396870 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:17.397326 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:17.397329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:17.397330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:17.397432 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:17.397623 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:17.397624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:17.397624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:17.397934 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:17.398083 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:17.398586 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 18:52:17.398691 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":117,"threads_started":1}
I20260811 18:52:17.400000 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:17.403780 19237 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:17.404507 19237 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:17.411062 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:17.403447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:34581 (local address 127.0.0.1:33021)
0811 18:52:17.403519 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:17.403521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:17.403741 (+ 220us) server_negotiation.cc:407] Connection header received
0811 18:52:17.403840 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:17.403841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:17.403854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:17.403912 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:17.404562 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:17.405343 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:17.406342 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:17.406689 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:17.406760 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:17.408216 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:17.408217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:17.408218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:17.408226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:17.409828 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:17.410228 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:17.410229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:17.410230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:17.410305 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:17.410498 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:17.410499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:17.410499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:17.410634 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:17.410745 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:17.410893 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 18:52:17.410962 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:52:17.416775 19237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ed45c6e8f054b6592a9a96e05c5ac5b: Generated new TSK 0
I20260811 18:52:17.416985 19237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:17.422557 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19051
W20260811 18:52:17.448216 19026 connection.cc:535] client connection to 127.0.0.1:33021 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33021): Connection reset by peer (error 104)
2026-08-11T18:52:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:17 dist-test-slave-pww0 krb5kdc[19255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:17 dist-test-slave-pww0 krb5kdc[19255](info): set up 2 sockets
Aug 11 18:52:17 dist-test-slave-pww0 krb5kdc[19255](info): commencing operation
krb5kdc: starting...
W20260811 18:52:18.675439 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:18 dist-test-slave-pww0 krb5kdc[19255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:18.804244 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:18.890143 19277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:18.890439 19277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:18.890475 19277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:18.891789 19277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:18.891844 19277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:18.891858 19277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:18.891872 19277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:18.891884 19277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:18.893498 19277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:18.893736 19277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:18.893945 19277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:18.898968 19287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:18.899120 19292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:18.898973 19288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:18.899380 19277 server_base.cc:1034] running on GCE node
I20260811 18:52:18.899683 19277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:18.900018 19277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:18.901175 19277 hybrid_clock.cc:648] HybridClock initialized: now 1786474338901163 us; error 35 us; skew 500 ppm
Aug 11 18:52:18 dist-test-slave-pww0 krb5kdc[19255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474338, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:18.903844 19277 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:18.904323 19277 webserver.cc:460] Webserver started at http://127.0.0.1:34269/ using document root <none> and password file <none>
I20260811 18:52:18.904587 19277 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:18.904644 19277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:18.904778 19277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:18.905901 19277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3"
format_stamp: "Formatted at 2026-08-11 18:52:18 on dist-test-slave-pww0"
server_key: "79d2cd634d5505486894e6b8afca7e69"
server_key_iv: "535ad3c5d259a9ac7c79927de1b0748e"
server_key_version: "encryptionkey@0"
I20260811 18:52:18.906270 19277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3"
format_stamp: "Formatted at 2026-08-11 18:52:18 on dist-test-slave-pww0"
server_key: "79d2cd634d5505486894e6b8afca7e69"
server_key_iv: "535ad3c5d259a9ac7c79927de1b0748e"
server_key_version: "encryptionkey@0"
I20260811 18:52:18.907766 19277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:52:18.908757 19300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:18.908928 19277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:18.909037 19277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3"
format_stamp: "Formatted at 2026-08-11 18:52:18 on dist-test-slave-pww0"
server_key: "79d2cd634d5505486894e6b8afca7e69"
server_key_iv: "535ad3c5d259a9ac7c79927de1b0748e"
server_key_version: "encryptionkey@0"
I20260811 18:52:18.909121 19277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:18.925206 19277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:18.927353 19277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:18.927582 19277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:18.932075 19277 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39693
I20260811 18:52:18.932125 19416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39693 every 8 connection(s)
I20260811 18:52:18.932493 19277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:18.933445 19417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:18.935861 19417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3: Bootstrap starting.
I20260811 18:52:18.936455 19417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:18.936725 19417 log.cc:824] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:18.937445 19417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3: No bootstrap required, opened a new log
I20260811 18:52:18.939172 19417 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39693 } }
I20260811 18:52:18.939323 19417 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:18.939345 19417 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d7e88bb2194f678d7ebfb27ccafbc3, State: Initialized, Role: FOLLOWER
I20260811 18:52:18.939510 19417 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39693 } }
I20260811 18:52:18.939584 19417 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:18.939613 19417 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:18.939666 19417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:18.940227 19417 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39693 } }
I20260811 18:52:18.940327 19417 leader_election.cc:302] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d7e88bb2194f678d7ebfb27ccafbc3; no voters:
I20260811 18:52:18.940363 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19277
I20260811 18:52:18.940477 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:18.940513 19417 leader_election.cc:288] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:18.940554 19423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:18.940694 19423 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [term 1 LEADER]: Becoming Leader. State: Replica: a2d7e88bb2194f678d7ebfb27ccafbc3, State: Running, Role: LEADER
I20260811 18:52:18.940706 10076 external_mini_cluster.cc:1428] Setting key 53f8e749677f2f6242becc9285e05443
I20260811 18:52:18.940838 19423 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39693 } }
I20260811 18:52:18.940867 19417 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:18.941376 19424 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2d7e88bb2194f678d7ebfb27ccafbc3. Latest consensus state: current_term: 1 leader_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39693 } } }
I20260811 18:52:18.941465 19424 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:18.941671 19438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:18.941622 19429 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d7e88bb2194f678d7ebfb27ccafbc3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39693 } } }
I20260811 18:52:18.941941 19429 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:18.942204 19438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:18.943910 19438 catalog_manager.cc:1347] Generated new cluster ID: f39e023590814886b342da582f8e3390
I20260811 18:52:18.943972 19438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:18.949141 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:18.941335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:55113 (local address 127.0.0.1:39693)
0811 18:52:18.941535 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:18.941546 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:18.943181 (+ 1635us) server_negotiation.cc:407] Connection header received
0811 18:52:18.943358 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:18.943367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:18.943433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:18.943616 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:18.943967 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:18.945444 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:18.946656 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:18.947373 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:18.947533 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:18.948369 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:18.948374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:18.948374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:18.948604 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:18.948646 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:18.948662 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:18.948745 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:18.948852 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 18:52:18.948883 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260811 18:52:18.950205 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:18.953819 19438 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:18.954517 19438 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:18.955502 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:18.951734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:51381 (local address 127.0.0.1:39693)
0811 18:52:18.951776 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:18.951778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:18.951870 (+ 92us) server_negotiation.cc:407] Connection header received
0811 18:52:18.951921 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:18.951922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:18.951932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:18.951983 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:18.952114 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:18.952960 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:18.954028 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:18.954347 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:18.954418 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:18.955120 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:18.955121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:18.955122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:18.955127 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:18.955143 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:18.955147 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:18.955200 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:18.955275 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 18:52:18.955290 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":22}
I20260811 18:52:18.959928 19438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2d7e88bb2194f678d7ebfb27ccafbc3: Generated new TSK 0
I20260811 18:52:18.960139 19438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:18.966797 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19277
W20260811 18:52:19.002271 19250 connection.cc:535] client connection to 127.0.0.1:39693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39693): Connection reset by peer (error 104)
2026-08-11T18:52:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:19 dist-test-slave-pww0 krb5kdc[19483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:19 dist-test-slave-pww0 krb5kdc[19483](info): set up 2 sockets
Aug 11 18:52:19 dist-test-slave-pww0 krb5kdc[19483](info): commencing operation
krb5kdc: starting...
W20260811 18:52:20.232172 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:20 dist-test-slave-pww0 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:20.370932 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:20.457306 19502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:20.457556 19502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:20.457640 19502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:20.458961 19502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:20.459017 19502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:20.459038 19502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:20.459054 19502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:20.459069 19502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:20.460808 19502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:20.461100 19502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:20.461341 19502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:52:20.466892 19502 server_base.cc:1034] running on GCE node
W20260811 18:52:20.466830 19519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:20.466830 19517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:20.466856 19516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:20.467374 19502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:20.467743 19502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:20.468920 19502 hybrid_clock.cc:648] HybridClock initialized: now 1786474340468902 us; error 33 us; skew 500 ppm
Aug 11 18:52:20 dist-test-slave-pww0 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474340, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:20.471858 19502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:20.472394 19502 webserver.cc:460] Webserver started at http://127.0.0.1:38709/ using document root <none> and password file <none>
I20260811 18:52:20.472615 19502 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:20.472663 19502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:20.472785 19502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:20.473927 19502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "c0bacac85009495e96f80096926bf471"
format_stamp: "Formatted at 2026-08-11 18:52:20 on dist-test-slave-pww0"
server_key: "8e5880ce22638870606ec1683dfa634d"
server_key_iv: "fbbd33f9bebc1d8fe5d853d32937c308"
server_key_version: "encryptionkey@0"
I20260811 18:52:20.474287 19502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "c0bacac85009495e96f80096926bf471"
format_stamp: "Formatted at 2026-08-11 18:52:20 on dist-test-slave-pww0"
server_key: "8e5880ce22638870606ec1683dfa634d"
server_key_iv: "fbbd33f9bebc1d8fe5d853d32937c308"
server_key_version: "encryptionkey@0"
I20260811 18:52:20.475767 19502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:20.476696 19532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:20.476864 19502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:20.476958 19502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c0bacac85009495e96f80096926bf471"
format_stamp: "Formatted at 2026-08-11 18:52:20 on dist-test-slave-pww0"
server_key: "8e5880ce22638870606ec1683dfa634d"
server_key_iv: "fbbd33f9bebc1d8fe5d853d32937c308"
server_key_version: "encryptionkey@0"
I20260811 18:52:20.477065 19502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:20.480706 19502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:20.483174 19502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:20.483325 19502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:20.487529 19502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39427
I20260811 18:52:20.487560 19631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39427 every 8 connection(s)
I20260811 18:52:20.487926 19502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:20.488965 19632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:20.491401 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471: Bootstrap starting.
I20260811 18:52:20.492056 19632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:20.492316 19632 log.cc:824] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:20.493127 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471: No bootstrap required, opened a new log
I20260811 18:52:20.494848 19632 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bacac85009495e96f80096926bf471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39427 } }
I20260811 18:52:20.495013 19632 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:20.495034 19632 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bacac85009495e96f80096926bf471, State: Initialized, Role: FOLLOWER
I20260811 18:52:20.495163 19632 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bacac85009495e96f80096926bf471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39427 } }
I20260811 18:52:20.495246 19632 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:20.495275 19632 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:20.495313 19632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:20.495957 19632 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bacac85009495e96f80096926bf471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39427 } }
I20260811 18:52:20.496052 19632 leader_election.cc:302] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0bacac85009495e96f80096926bf471; no voters:
I20260811 18:52:20.496192 19632 leader_election.cc:288] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:20.496275 19637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:20.496450 19637 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [term 1 LEADER]: Becoming Leader. State: Replica: c0bacac85009495e96f80096926bf471, State: Running, Role: LEADER
I20260811 18:52:20.496490 19632 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:20.496548 19637 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bacac85009495e96f80096926bf471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39427 } }
I20260811 18:52:20.496968 19639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bacac85009495e96f80096926bf471. Latest consensus state: current_term: 1 leader_uuid: "c0bacac85009495e96f80096926bf471" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bacac85009495e96f80096926bf471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39427 } } }
I20260811 18:52:20.497056 19639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:20.497157 19638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0bacac85009495e96f80096926bf471" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bacac85009495e96f80096926bf471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39427 } } }
I20260811 18:52:20.497256 19638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:20.497430 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19502
I20260811 18:52:20.497565 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:20.497794 10076 external_mini_cluster.cc:1428] Setting key a472aae40849a25a4a44eb4217d04967
W20260811 18:52:20.498389 19663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:20.498446 19663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:20.498498 19650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:20.498683 19650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:20.500458 19650 catalog_manager.cc:1347] Generated new cluster ID: 463b2c9465b142999f9b39ff5c6414d6
I20260811 18:52:20.500524 19650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:20.501538 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:20.498324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:52695 (local address 127.0.0.1:39427)
0811 18:52:20.499852 (+ 1528us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:20.499863 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:20.499901 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:52:20.500022 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:20.500028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:20.500053 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:20.500173 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:20.501062 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:20.501065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:20.501065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:20.501068 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:20.501104 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:20.501115 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:20.501171 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:20.501245 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 18:52:20.501382 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1463,"thread_start_us":63,"threads_started":1}
I20260811 18:52:20.502568 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:20.510004 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:20.504351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:48617 (local address 127.0.0.1:39427)
0811 18:52:20.504398 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:20.504400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:20.507804 (+ 3404us) server_negotiation.cc:407] Connection header received
0811 18:52:20.507838 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:20.507839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:20.507848 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:20.507889 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:20.509770 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:20.509772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:20.509773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:20.509779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:20.509801 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:20.509807 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:20.509860 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:20.509901 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 18:52:20.509919 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:52:20.536899 19650 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:20.537122 19650 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:20.542711 19650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0bacac85009495e96f80096926bf471: Generated new TSK 0
I20260811 18:52:20.543058 19650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:20.550827 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19502
2026-08-11T18:52:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:20 dist-test-slave-pww0 krb5kdc[19688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:20 dist-test-slave-pww0 krb5kdc[19688](info): set up 2 sockets
Aug 11 18:52:20 dist-test-slave-pww0 krb5kdc[19688](info): commencing operation
krb5kdc: starting...
W20260811 18:52:21.800843 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:21 dist-test-slave-pww0 krb5kdc[19688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:21.932173 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:22.017831 19715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:22.018091 19715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:22.018134 19715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:22.019759 19715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:22.019822 19715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:22.019842 19715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:22.019860 19715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:22.019877 19715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:22.021576 19715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:22.021874 19715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:22.022102 19715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:22.027366 19729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:22.027423 19725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:22.027374 19727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:22.027475 19715 server_base.cc:1034] running on GCE node
I20260811 18:52:22.027964 19715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:22.028286 19715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:22.029462 19715 hybrid_clock.cc:648] HybridClock initialized: now 1786474342029439 us; error 38 us; skew 500 ppm
Aug 11 18:52:22 dist-test-slave-pww0 krb5kdc[19688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474342, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:22.035631 19715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:22.036170 19715 webserver.cc:460] Webserver started at http://127.0.0.1:38513/ using document root <none> and password file <none>
I20260811 18:52:22.036396 19715 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:22.036449 19715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:22.036583 19715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:22.037714 19715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "38e44ba33af2458c9f69d5a96d4611cd"
format_stamp: "Formatted at 2026-08-11 18:52:22 on dist-test-slave-pww0"
server_key: "bf08e238d276bae436d025db16669e76"
server_key_iv: "efbdaef84b906ec7b52a16b0effc8869"
server_key_version: "encryptionkey@0"
I20260811 18:52:22.038064 19715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "38e44ba33af2458c9f69d5a96d4611cd"
format_stamp: "Formatted at 2026-08-11 18:52:22 on dist-test-slave-pww0"
server_key: "bf08e238d276bae436d025db16669e76"
server_key_iv: "efbdaef84b906ec7b52a16b0effc8869"
server_key_version: "encryptionkey@0"
I20260811 18:52:22.039513 19715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:22.040390 19744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:22.040611 19715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:22.040725 19715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "38e44ba33af2458c9f69d5a96d4611cd"
format_stamp: "Formatted at 2026-08-11 18:52:22 on dist-test-slave-pww0"
server_key: "bf08e238d276bae436d025db16669e76"
server_key_iv: "efbdaef84b906ec7b52a16b0effc8869"
server_key_version: "encryptionkey@0"
I20260811 18:52:22.040795 19715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:22.066805 19715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:22.069178 19715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:22.069347 19715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:22.073596 19715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39669
I20260811 18:52:22.073648 19844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39669 every 8 connection(s)
I20260811 18:52:22.074021 19715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:22.074935 19846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:22.077387 19846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd: Bootstrap starting.
I20260811 18:52:22.078007 19846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:22.078284 19846 log.cc:824] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:22.078701 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19715
I20260811 18:52:22.078812 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:22.078994 10076 external_mini_cluster.cc:1428] Setting key 9522c812f85c90ce1cfa0ff13c4cb45c
I20260811 18:52:22.079147 19846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd: No bootstrap required, opened a new log
I20260811 18:52:22.081508 19846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e44ba33af2458c9f69d5a96d4611cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260811 18:52:22.081670 19846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:22.081702 19846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e44ba33af2458c9f69d5a96d4611cd, State: Initialized, Role: FOLLOWER
I20260811 18:52:22.081841 19846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e44ba33af2458c9f69d5a96d4611cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260811 18:52:22.081924 19846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:22.081956 19846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:22.082005 19846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:22.082641 19846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e44ba33af2458c9f69d5a96d4611cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260811 18:52:22.082749 19846 leader_election.cc:302] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38e44ba33af2458c9f69d5a96d4611cd; no voters:
I20260811 18:52:22.082924 19846 leader_election.cc:288] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:22.082979 19854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:22.083176 19854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [term 1 LEADER]: Becoming Leader. State: Replica: 38e44ba33af2458c9f69d5a96d4611cd, State: Running, Role: LEADER
I20260811 18:52:22.083284 19846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:22.083324 19854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e44ba33af2458c9f69d5a96d4611cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260811 18:52:22.083714 19856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38e44ba33af2458c9f69d5a96d4611cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e44ba33af2458c9f69d5a96d4611cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } } }
I20260811 18:52:22.083815 19856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:22.083725 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38e44ba33af2458c9f69d5a96d4611cd. Latest consensus state: current_term: 1 leader_uuid: "38e44ba33af2458c9f69d5a96d4611cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e44ba33af2458c9f69d5a96d4611cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } } }
I20260811 18:52:22.083868 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:22.085074 19891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:22.085129 19891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:22.085239 19867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:22.085441 19867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 18:52:22 dist-test-slave-pww0 krb5kdc[19688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:52:22.087883 19867 catalog_manager.cc:1347] Generated new cluster ID: 6605bd00f808456d97d6781ae509c9b9
I20260811 18:52:22.088346 19867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:22.091132 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:22.079606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:36897 (local address 127.0.0.1:39669)
0811 18:52:22.079776 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:22.079785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:22.079813 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:52:22.080502 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:22.080511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:22.080577 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:22.080701 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:22.081524 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:22.082870 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:22.084019 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:22.084702 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:22.084800 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:22.087730 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:22.087735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:22.087735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:22.087750 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:22.089473 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:22.089925 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:22.089929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:22.089930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:22.090022 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:22.090252 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:22.090253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:22.090253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:22.090515 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:22.090698 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:22.090798 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 18:52:22.090904 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260811 18:52:22.092134 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:22.099483 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:22.093502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:52789 (local address 127.0.0.1:39669)
0811 18:52:22.093542 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:22.093544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:22.093661 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:52:22.093716 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:22.093717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:22.093727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:22.093772 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:22.094270 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:22.094782 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:22.095504 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:22.095778 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:22.095836 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:22.097027 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:22.097029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:22.097029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:22.097038 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:22.098330 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:22.098690 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:22.098691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:22.098691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:22.098743 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:22.098936 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:22.098937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:22.098937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:22.099050 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:22.099159 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:22.099298 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 18:52:22.099362 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:22.117381 19867 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:22.117974 19867 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:22.121924 19867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38e44ba33af2458c9f69d5a96d4611cd: Generated new TSK 0
I20260811 18:52:22.122128 19867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:22.129998 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19715
W20260811 18:52:22.140730 19685 connection.cc:535] client connection to 127.0.0.1:39669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39669): Connection reset by peer (error 104)
2026-08-11T18:52:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:22 dist-test-slave-pww0 krb5kdc[19905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:22 dist-test-slave-pww0 krb5kdc[19905](info): set up 2 sockets
Aug 11 18:52:22 dist-test-slave-pww0 krb5kdc[19905](info): commencing operation
krb5kdc: starting...
W20260811 18:52:23.370991 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:23 dist-test-slave-pww0 krb5kdc[19905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:23.499591 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:23.586019 19927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:23.586277 19927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:23.586315 19927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:23.587682 19927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:23.587751 19927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:23.587770 19927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:23.587787 19927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:23.587801 19927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:23.589515 19927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34653
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:23.589819 19927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:23.590054 19927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:23.595085 19938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:23.595085 19933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:23.595343 19927 server_base.cc:1034] running on GCE node
W20260811 18:52:23.595474 19936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:23.595800 19927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:23.596082 19927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:23.597244 19927 hybrid_clock.cc:648] HybridClock initialized: now 1786474343597224 us; error 32 us; skew 500 ppm
Aug 11 18:52:23 dist-test-slave-pww0 krb5kdc[19905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474343, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:23.599854 19927 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:23.600337 19927 webserver.cc:460] Webserver started at http://127.0.0.1:41725/ using document root <none> and password file <none>
I20260811 18:52:23.600550 19927 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:23.600596 19927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:23.600709 19927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:23.601795 19927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "8f3a0268b6504e54ac21fdbcc75bbed9"
format_stamp: "Formatted at 2026-08-11 18:52:23 on dist-test-slave-pww0"
server_key: "8b8b3597eef1964e46336ab7aae72e5c"
server_key_iv: "a624b00df593bf8ad3b9154a8c195007"
server_key_version: "encryptionkey@0"
I20260811 18:52:23.602128 19927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "8f3a0268b6504e54ac21fdbcc75bbed9"
format_stamp: "Formatted at 2026-08-11 18:52:23 on dist-test-slave-pww0"
server_key: "8b8b3597eef1964e46336ab7aae72e5c"
server_key_iv: "a624b00df593bf8ad3b9154a8c195007"
server_key_version: "encryptionkey@0"
I20260811 18:52:23.603493 19927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 18:52:23.604332 19954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:23.604518 19927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:23.604617 19927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "8f3a0268b6504e54ac21fdbcc75bbed9"
format_stamp: "Formatted at 2026-08-11 18:52:23 on dist-test-slave-pww0"
server_key: "8b8b3597eef1964e46336ab7aae72e5c"
server_key_iv: "a624b00df593bf8ad3b9154a8c195007"
server_key_version: "encryptionkey@0"
I20260811 18:52:23.604689 19927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:23.616542 19927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:23.618404 19927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:23.618558 19927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:23.622654 19927 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34653
I20260811 18:52:23.622691 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34653 every 8 connection(s)
I20260811 18:52:23.623059 19927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:23.623980 20066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:23.625705 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 19927
I20260811 18:52:23.625838 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:23.626044 10076 external_mini_cluster.cc:1428] Setting key a1a11fbdc4dbbc646c19409d80cd0476
I20260811 18:52:23.626953 20066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9: Bootstrap starting.
I20260811 18:52:23.627656 20066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:23.628005 20066 log.cc:824] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:23.628718 20066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9: No bootstrap required, opened a new log
I20260811 18:52:23.631136 20066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 18:52:23.631340 20066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:23.631371 20066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3a0268b6504e54ac21fdbcc75bbed9, State: Initialized, Role: FOLLOWER
I20260811 18:52:23.631520 20066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 18:52:23.631598 20066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:23.631623 20066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:23.631668 20066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:23.632133 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:23.626662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:47497 (local address 127.0.0.1:34653)
0811 18:52:23.626843 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:23.626852 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:23.626882 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:52:23.627377 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:23.627386 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:23.627445 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:23.627619 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:23.627862 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:23.628867 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:23.629978 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:23.630424 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:23.630510 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:23.631301 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:23.631305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:23.631305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:23.631627 (+ 322us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:23.631676 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:23.631693 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:23.631774 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:23.631884 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:52:23.631912 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260811 18:52:23.632380 20066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 18:52:23.632494 20066 leader_election.cc:302] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f3a0268b6504e54ac21fdbcc75bbed9; no voters:
I20260811 18:52:23.632665 20066 leader_election.cc:288] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:23.632707 20076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:23.632865 20076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [term 1 LEADER]: Becoming Leader. State: Replica: 8f3a0268b6504e54ac21fdbcc75bbed9, State: Running, Role: LEADER
I20260811 18:52:23.632944 20066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:23.633008 20076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } }
I20260811 18:52:23.633232 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:23.633410 20078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } } }
I20260811 18:52:23.633491 20078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:23.633756 20079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f3a0268b6504e54ac21fdbcc75bbed9. Latest consensus state: current_term: 1 leader_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3a0268b6504e54ac21fdbcc75bbed9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34653 } } }
I20260811 18:52:23.634056 20079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:23.634220 20089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:23.634970 20089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:23.636631 20089 catalog_manager.cc:1347] Generated new cluster ID: 13903e93884a48f28e92fffe00177961
I20260811 18:52:23.636680 20089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:23.649376 20089 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:23.649699 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:23.634879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:34203 (local address 127.0.0.1:34653)
0811 18:52:23.634921 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:23.634923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:23.635030 (+ 107us) server_negotiation.cc:407] Connection header received
0811 18:52:23.635079 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:23.635080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:23.635092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:23.635153 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:23.635302 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:23.635952 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:23.647983 (+ 12031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:23.648326 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:23.648412 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:23.649406 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:23.649407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:23.649408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:23.649414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:23.649433 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:23.649437 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:23.649472 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:23.649606 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 18:52:23.649618 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:23.650401 20089 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:23.667964 20089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f3a0268b6504e54ac21fdbcc75bbed9: Generated new TSK 0
I20260811 18:52:23.668226 20089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:23.673964 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 19927
W20260811 18:52:23.689474 19900 connection.cc:535] client connection to 127.0.0.1:34653 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34653): Connection reset by peer (error 104)
2026-08-11T18:52:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:23 dist-test-slave-pww0 krb5kdc[20124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:23 dist-test-slave-pww0 krb5kdc[20124](info): set up 2 sockets
Aug 11 18:52:23 dist-test-slave-pww0 krb5kdc[20124](info): commencing operation
krb5kdc: starting...
W20260811 18:52:24.909756 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:25 dist-test-slave-pww0 krb5kdc[20124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:52:25.048424 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:25.136641 20141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:25.136873 20141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:25.136899 20141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:25.138196 20141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:25.138262 20141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:25.138274 20141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:25.138286 20141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:25.138298 20141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:25.140057 20141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:25.140367 20141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:25.140682 20141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:25.146045 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:25.146092 20155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:25.146199 20141 server_base.cc:1034] running on GCE node
W20260811 18:52:25.146049 20152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:25.146623 20141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:25.146958 20141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:25.148144 20141 hybrid_clock.cc:648] HybridClock initialized: now 1786474345148124 us; error 39 us; skew 500 ppm
Aug 11 18:52:25 dist-test-slave-pww0 krb5kdc[20124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474345, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:25.151081 20141 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:52:25.151651 20141 webserver.cc:460] Webserver started at http://127.0.0.1:41317/ using document root <none> and password file <none>
I20260811 18:52:25.151885 20141 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:25.151939 20141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:25.152084 20141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:25.153223 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "1e1ab03193254ecc96c46887dbfcf5e3"
format_stamp: "Formatted at 2026-08-11 18:52:25 on dist-test-slave-pww0"
server_key: "958cc64a7b88916a4ea8460307547e5b"
server_key_iv: "93b937ad4163c5e9c87877f248f62d16"
server_key_version: "encryptionkey@0"
I20260811 18:52:25.153582 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "1e1ab03193254ecc96c46887dbfcf5e3"
format_stamp: "Formatted at 2026-08-11 18:52:25 on dist-test-slave-pww0"
server_key: "958cc64a7b88916a4ea8460307547e5b"
server_key_iv: "93b937ad4163c5e9c87877f248f62d16"
server_key_version: "encryptionkey@0"
I20260811 18:52:25.155138 20141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:25.156148 20170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:25.156339 20141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:25.156442 20141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "1e1ab03193254ecc96c46887dbfcf5e3"
format_stamp: "Formatted at 2026-08-11 18:52:25 on dist-test-slave-pww0"
server_key: "958cc64a7b88916a4ea8460307547e5b"
server_key_iv: "93b937ad4163c5e9c87877f248f62d16"
server_key_version: "encryptionkey@0"
I20260811 18:52:25.156517 20141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:25.160147 20141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:25.162551 20141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:25.162704 20141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:25.166941 20141 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39897
I20260811 18:52:25.166989 20273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39897 every 8 connection(s)
I20260811 18:52:25.167362 20141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:25.168514 20274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:25.170994 20274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3: Bootstrap starting.
I20260811 18:52:25.171691 20274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:25.171964 20274 log.cc:824] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:25.172838 20274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3: No bootstrap required, opened a new log
I20260811 18:52:25.174582 20274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260811 18:52:25.174772 20274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:25.174795 20274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e1ab03193254ecc96c46887dbfcf5e3, State: Initialized, Role: FOLLOWER
I20260811 18:52:25.174791 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20141
I20260811 18:52:25.174898 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:25.174942 20274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260811 18:52:25.175035 20274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:25.175105 10076 external_mini_cluster.cc:1428] Setting key bfa6ec6051a2bb4064826c292d7e5471
I20260811 18:52:25.175169 20274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:25.175264 20274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:25.176503 20274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260811 18:52:25.176637 20274 leader_election.cc:302] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e1ab03193254ecc96c46887dbfcf5e3; no voters:
I20260811 18:52:25.176823 20274 leader_election.cc:288] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:25.176869 20279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:25.176978 20279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1e1ab03193254ecc96c46887dbfcf5e3, State: Running, Role: LEADER
I20260811 18:52:25.177135 20274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:25.177135 20279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } }
I20260811 18:52:25.177486 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:25.175660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:35185 (local address 127.0.0.1:39897)
0811 18:52:25.175868 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:25.175878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:25.175906 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:52:25.176024 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:25.176030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:25.176065 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:25.176183 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:25.176861 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:25.176866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:25.176866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:25.176871 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:25.176926 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:25.176942 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:25.177005 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:25.177245 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 18:52:25.177297 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260811 18:52:25.178231 20280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260811 18:52:25.178321 20280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:25.178574 20281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e1ab03193254ecc96c46887dbfcf5e3. Latest consensus state: current_term: 1 leader_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ab03193254ecc96c46887dbfcf5e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39897 } } }
I20260811 18:52:25.178678 20281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:25.178658 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:25.178810 20295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:25.179589 20295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:25.181692 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:25.180532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:45299 (local address 127.0.0.1:39897)
0811 18:52:25.180586 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:25.180587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:25.180598 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:52:25.180685 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:25.180686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:25.180692 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:25.180752 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:25.181389 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:25.181391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:25.181391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:25.181397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:25.181414 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:25.181419 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:25.181472 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:25.181613 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 18:52:25.181628 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:52:25.181773 20295 catalog_manager.cc:1347] Generated new cluster ID: be7715393af84d2993ade72f7ca55873
I20260811 18:52:25.181814 20295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:25.195570 20295 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:25.195904 20295 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:25.200579 20295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e1ab03193254ecc96c46887dbfcf5e3: Generated new TSK 0
I20260811 18:52:25.200851 20295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:25.206789 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20141
2026-08-11T18:52:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:25 dist-test-slave-pww0 krb5kdc[20319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:25 dist-test-slave-pww0 krb5kdc[20319](info): set up 2 sockets
Aug 11 18:52:25 dist-test-slave-pww0 krb5kdc[20319](info): commencing operation
krb5kdc: starting...
W20260811 18:52:26.446182 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:26 dist-test-slave-pww0 krb5kdc[20319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:26.578817 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:41113
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:40961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:41113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:26.662449 20355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:26.662686 20355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:26.662721 20355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:26.663961 20355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:26.664002 20355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:26.664016 20355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:26.664027 20355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:26.664037 20355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:26.665624 20355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:40961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:41113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:41113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:26.665886 20355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:26.666096 20355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:26.671295 20376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:26.671393 20368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:26.671504 20372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:26.671442 20355 server_base.cc:1034] running on GCE node
I20260811 18:52:26.671809 20355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:26.672125 20355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:26.673293 20355 hybrid_clock.cc:648] HybridClock initialized: now 1786474346673275 us; error 31 us; skew 500 ppm
Aug 11 18:52:26 dist-test-slave-pww0 krb5kdc[20319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474346, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:26.676020 20355 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:26.676484 20355 webserver.cc:460] Webserver started at http://127.9.215.62:34359/ using document root <none> and password file <none>
I20260811 18:52:26.676692 20355 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:26.676740 20355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:26.676833 20355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:26.677917 20355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "fb2091ba9c304dadafdffa2da5e933cb"
format_stamp: "Formatted at 2026-08-11 18:52:26 on dist-test-slave-pww0"
server_key: "f2099bc14fdce8ada689297e86d7a274"
server_key_iv: "9b410f1ffdd93ec4526fbfb214820efc"
server_key_version: "encryptionkey@0"
I20260811 18:52:26.678407 20355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "fb2091ba9c304dadafdffa2da5e933cb"
format_stamp: "Formatted at 2026-08-11 18:52:26 on dist-test-slave-pww0"
server_key: "f2099bc14fdce8ada689297e86d7a274"
server_key_iv: "9b410f1ffdd93ec4526fbfb214820efc"
server_key_version: "encryptionkey@0"
I20260811 18:52:26.679822 20355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:26.680588 20393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:26.680783 20355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:26.680879 20355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "fb2091ba9c304dadafdffa2da5e933cb"
format_stamp: "Formatted at 2026-08-11 18:52:26 on dist-test-slave-pww0"
server_key: "f2099bc14fdce8ada689297e86d7a274"
server_key_iv: "9b410f1ffdd93ec4526fbfb214820efc"
server_key_version: "encryptionkey@0"
I20260811 18:52:26.680944 20355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:26.694351 20355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:26.696650 20355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:26.696808 20355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:26.700896 20355 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:41113
I20260811 18:52:26.700966 20495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:41113 every 8 connection(s)
I20260811 18:52:26.701284 20355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:26.702188 20498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:26.704685 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20355
I20260811 18:52:26.704821 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:26.704843 20498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb: Bootstrap starting.
I20260811 18:52:26.705008 10076 external_mini_cluster.cc:1428] Setting key d823b1eb65f6c2878ca30354acfd885e
I20260811 18:52:26.705540 20498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:26.705909 20498 log.cc:824] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:26.706589 20498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb: No bootstrap required, opened a new log
I20260811 18:52:26.708606 20498 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2091ba9c304dadafdffa2da5e933cb" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 41113 } }
I20260811 18:52:26.708775 20498 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:26.708810 20498 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2091ba9c304dadafdffa2da5e933cb, State: Initialized, Role: FOLLOWER
I20260811 18:52:26.708956 20498 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2091ba9c304dadafdffa2da5e933cb" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 41113 } }
I20260811 18:52:26.709034 20498 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:26.709067 20498 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:26.709118 20498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:26.709782 20498 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2091ba9c304dadafdffa2da5e933cb" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 41113 } }
I20260811 18:52:26.709895 20498 leader_election.cc:302] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb2091ba9c304dadafdffa2da5e933cb; no voters:
I20260811 18:52:26.710076 20498 leader_election.cc:288] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:26.710253 20508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:26.710394 20498 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:26.710446 20508 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [term 1 LEADER]: Becoming Leader. State: Replica: fb2091ba9c304dadafdffa2da5e933cb, State: Running, Role: LEADER
I20260811 18:52:26.710558 20508 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2091ba9c304dadafdffa2da5e933cb" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 41113 } }
I20260811 18:52:26.711117 20507 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb2091ba9c304dadafdffa2da5e933cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2091ba9c304dadafdffa2da5e933cb" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 41113 } } }
I20260811 18:52:26.711218 20507 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:26.711390 20509 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb2091ba9c304dadafdffa2da5e933cb. Latest consensus state: current_term: 1 leader_uuid: "fb2091ba9c304dadafdffa2da5e933cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2091ba9c304dadafdffa2da5e933cb" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 41113 } } }
I20260811 18:52:26.711441 20509 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb [sys.catalog]: This master's current role is: LEADER
Aug 11 18:52:26 dist-test-slave-pww0 krb5kdc[20319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
W20260811 18:52:26.712504 20545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:26.712564 20545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:26.712620 20525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:26.712824 20525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:26.714493 20525 catalog_manager.cc:1347] Generated new cluster ID: 8c2000b1aa5e4598ac2c5c9ef7a79da5
I20260811 18:52:26.714550 20525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:26.716588 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:26.705609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56483 (local address 127.9.215.62:41113)
0811 18:52:26.705809 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:26.705818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:26.705846 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:52:26.706360 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:26.706368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:26.706421 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:26.706570 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:26.707239 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:26.708202 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:26.709011 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:26.709515 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:26.709632 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:26.712710 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:26.712715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:26.712716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:26.712733 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:26.714744 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:26.715245 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:26.715249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:26.715250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:26.715365 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:26.715621 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:26.715623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:26.715623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:26.715960 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:26.716140 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:26.716246 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 18:52:26.716359 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260811 18:52:26.717600 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:26.725414 20525 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:26.725932 20525 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:26.727012 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:26.719391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52203 (local address 127.9.215.62:41113)
0811 18:52:26.719432 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:26.719434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:26.719511 (+ 77us) server_negotiation.cc:407] Connection header received
0811 18:52:26.719582 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:26.719583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:26.719593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:26.719645 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:26.720261 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:26.721044 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:26.722021 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:26.722366 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:26.722441 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:26.723869 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:26.723870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:26.723871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:26.723879 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:26.725503 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:26.725952 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:26.725954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:26.725955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:26.726012 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:26.726244 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:26.726245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:26.726246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:26.726386 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:26.726477 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:26.726857 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 18:52:26.726919 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":19}
I20260811 18:52:26.728210 20525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb2091ba9c304dadafdffa2da5e933cb: Generated new TSK 0
I20260811 18:52:26.728390 20525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:26.730600 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20355
W20260811 18:52:26.754451 20317 connection.cc:535] client connection to 127.9.215.62:41113 recv error: Network error: failed to read from TLS socket (remote: 127.9.215.62:41113): Connection reset by peer (error 104)
2026-08-11T18:52:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:26 dist-test-slave-pww0 krb5kdc[20559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:26 dist-test-slave-pww0 krb5kdc[20559](info): set up 2 sockets
Aug 11 18:52:26 dist-test-slave-pww0 krb5kdc[20559](info): commencing operation
krb5kdc: starting...
W20260811 18:52:27.968726 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:28 dist-test-slave-pww0 krb5kdc[20559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:28.102486 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36361
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:37581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:28.198027 20587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:28.198284 20587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:28.198319 20587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:28.199651 20587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:28.199705 20587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:28.199726 20587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:28.199743 20587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:28.199759 20587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:28.201467 20587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:37581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36361
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:28.201766 20587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:28.202000 20587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:28.207772 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:28.207782 20606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:28.207844 20600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:28.207960 20587 server_base.cc:1034] running on GCE node
I20260811 18:52:28.208292 20587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:28.208632 20587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:28.209794 20587 hybrid_clock.cc:648] HybridClock initialized: now 1786474348209777 us; error 43 us; skew 500 ppm
Aug 11 18:52:28 dist-test-slave-pww0 krb5kdc[20559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:28.212495 20587 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:28.212998 20587 webserver.cc:460] Webserver started at http://127.9.215.62:39409/ using document root <none> and password file <none>
I20260811 18:52:28.213213 20587 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:28.213264 20587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:28.213400 20587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:28.214515 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "12c35f53b5314d3e9a6e57da4dc06420"
format_stamp: "Formatted at 2026-08-11 18:52:28 on dist-test-slave-pww0"
server_key: "d0e0baf4bad01bcd5d7dc6d1f976eea9"
server_key_iv: "0deb193a9bb07cd5ffdd064f63a8fea7"
server_key_version: "encryptionkey@0"
I20260811 18:52:28.214884 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "12c35f53b5314d3e9a6e57da4dc06420"
format_stamp: "Formatted at 2026-08-11 18:52:28 on dist-test-slave-pww0"
server_key: "d0e0baf4bad01bcd5d7dc6d1f976eea9"
server_key_iv: "0deb193a9bb07cd5ffdd064f63a8fea7"
server_key_version: "encryptionkey@0"
I20260811 18:52:28.216398 20587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:28.217238 20619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:28.217401 20587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:28.217495 20587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "12c35f53b5314d3e9a6e57da4dc06420"
format_stamp: "Formatted at 2026-08-11 18:52:28 on dist-test-slave-pww0"
server_key: "d0e0baf4bad01bcd5d7dc6d1f976eea9"
server_key_iv: "0deb193a9bb07cd5ffdd064f63a8fea7"
server_key_version: "encryptionkey@0"
I20260811 18:52:28.217563 20587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:28.228686 20587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:28.231282 20587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:28.231431 20587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:28.235757 20587 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36361
I20260811 18:52:28.235790 20722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36361 every 8 connection(s)
I20260811 18:52:28.236171 20587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:28.237179 20726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:28.239679 20726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420: Bootstrap starting.
I20260811 18:52:28.240216 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20587
I20260811 18:52:28.240350 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:28.240401 20726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:28.240566 10076 external_mini_cluster.cc:1428] Setting key faca90de90fa31e77757ecfbd35cc483
I20260811 18:52:28.240754 20726 log.cc:824] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:28.241567 20726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420: No bootstrap required, opened a new log
I20260811 18:52:28.244057 20726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c35f53b5314d3e9a6e57da4dc06420" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36361 } }
I20260811 18:52:28.244261 20726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:28.244297 20726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12c35f53b5314d3e9a6e57da4dc06420, State: Initialized, Role: FOLLOWER
I20260811 18:52:28.244436 20726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c35f53b5314d3e9a6e57da4dc06420" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36361 } }
I20260811 18:52:28.244525 20726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:28.244568 20726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:28.244620 20726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:28.245326 20726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c35f53b5314d3e9a6e57da4dc06420" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36361 } }
I20260811 18:52:28.245450 20726 leader_election.cc:302] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12c35f53b5314d3e9a6e57da4dc06420; no voters:
I20260811 18:52:28.245656 20726 leader_election.cc:288] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:28.245682 20738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:28.245852 20738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [term 1 LEADER]: Becoming Leader. State: Replica: 12c35f53b5314d3e9a6e57da4dc06420, State: Running, Role: LEADER
I20260811 18:52:28.245962 20738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c35f53b5314d3e9a6e57da4dc06420" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36361 } }
I20260811 18:52:28.245972 20726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:28.246384 20739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12c35f53b5314d3e9a6e57da4dc06420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c35f53b5314d3e9a6e57da4dc06420" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36361 } } }
I20260811 18:52:28.246480 20739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:28.246671 20740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12c35f53b5314d3e9a6e57da4dc06420. Latest consensus state: current_term: 1 leader_uuid: "12c35f53b5314d3e9a6e57da4dc06420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12c35f53b5314d3e9a6e57da4dc06420" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36361 } } }
I20260811 18:52:28.246779 20740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:28.247511 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:28.241118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46099 (local address 127.9.215.62:36361)
0811 18:52:28.241309 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:28.241320 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:28.241355 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:52:28.241891 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:28.241900 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:28.241984 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:28.242113 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:28.242450 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:28.243395 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:28.244505 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:28.245028 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:28.245114 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:28.246695 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:28.246700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:28.246700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:28.246919 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:28.246955 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:28.246967 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:28.247045 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:28.247262 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 18:52:28.247296 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
W20260811 18:52:28.248054 20771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:28.248139 20771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:28.248209 20753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:28.248364 20753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:28.248559 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:28.250077 20753 catalog_manager.cc:1347] Generated new cluster ID: 68488b2f20504bbaac28c36bdf14e045
I20260811 18:52:28.250124 20753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:28.254033 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:28.250371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41607 (local address 127.9.215.62:36361)
0811 18:52:28.250415 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:28.250417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:28.250552 (+ 135us) server_negotiation.cc:407] Connection header received
0811 18:52:28.250618 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:28.250619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:28.250628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:28.250686 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:28.250842 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:28.251599 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:28.252661 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:28.252984 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:28.253037 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:28.253785 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:28.253786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:28.253787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:28.253792 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:28.253809 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:28.253814 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:28.253863 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:28.253939 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 18:52:28.253950 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:28.269840 20753 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:28.270679 20753 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:28.275084 20753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12c35f53b5314d3e9a6e57da4dc06420: Generated new TSK 0
I20260811 18:52:28.275321 20753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:28.281576 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20587
W20260811 18:52:28.309358 20555 connection.cc:535] client connection to 127.9.215.62:36361 recv error: Network error: failed to read from TLS socket (remote: 127.9.215.62:36361): Connection reset by peer (error 104)
2026-08-11T18:52:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:28 dist-test-slave-pww0 krb5kdc[20787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:28 dist-test-slave-pww0 krb5kdc[20787](info): set up 2 sockets
Aug 11 18:52:28 dist-test-slave-pww0 krb5kdc[20787](info): commencing operation
krb5kdc: starting...
W20260811 18:52:29.534844 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:29 dist-test-slave-pww0 krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:29.666467 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:42309
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:42309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:29.751736 20810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:29.751991 20810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:29.752027 20810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:29.753343 20810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:29.753404 20810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:29.753425 20810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:29.753441 20810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:29.753456 20810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:29.755143 20810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:42309
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:29.755439 20810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:29.755741 20810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:29.760855 20823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:29.760875 20819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:29.760882 20821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:29.761125 20810 server_base.cc:1034] running on GCE node
I20260811 18:52:29.761391 20810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:29.761704 20810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:29.762859 20810 hybrid_clock.cc:648] HybridClock initialized: now 1786474349762842 us; error 35 us; skew 500 ppm
Aug 11 18:52:29 dist-test-slave-pww0 krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:29.765671 20810 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:29.766129 20810 webserver.cc:460] Webserver started at http://127.9.215.62:33551/ using document root <none> and password file <none>
I20260811 18:52:29.766371 20810 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:29.766422 20810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:29.766552 20810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:29.767689 20810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "c2f573b3340d4dae97a4eb19daeccc81"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-pww0"
server_key: "ca68bf7178e9b257020741730958171a"
server_key_iv: "f65bebc7e3a1349150ef9045d96a91ea"
server_key_version: "encryptionkey@0"
I20260811 18:52:29.768025 20810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "c2f573b3340d4dae97a4eb19daeccc81"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-pww0"
server_key: "ca68bf7178e9b257020741730958171a"
server_key_iv: "f65bebc7e3a1349150ef9045d96a91ea"
server_key_version: "encryptionkey@0"
I20260811 18:52:29.769420 20810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:29.770238 20838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:29.770398 20810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:29.770493 20810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "c2f573b3340d4dae97a4eb19daeccc81"
format_stamp: "Formatted at 2026-08-11 18:52:29 on dist-test-slave-pww0"
server_key: "ca68bf7178e9b257020741730958171a"
server_key_iv: "f65bebc7e3a1349150ef9045d96a91ea"
server_key_version: "encryptionkey@0"
I20260811 18:52:29.770556 20810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:29.773986 20810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:29.776165 20810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:29.776320 20810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:29.780227 20810 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:42309
I20260811 18:52:29.780256 20955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:42309 every 8 connection(s)
I20260811 18:52:29.780603 20810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:29.781641 20956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:29.782743 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 20810
I20260811 18:52:29.782871 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:29.783057 10076 external_mini_cluster.cc:1428] Setting key e042955b52c3987d282d6b5923723d30
I20260811 18:52:29.784472 20956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81: Bootstrap starting.
I20260811 18:52:29.785207 20956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:29.785542 20956 log.cc:824] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:29.785772 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:29.783658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58129 (local address 127.9.215.62:42309)
0811 18:52:29.783863 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:29.783874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:29.783910 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:52:29.784456 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:29.784466 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:29.784504 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:29.784662 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:29.785260 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:29.785264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:29.785265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:29.785270 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:29.785336 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:29.785354 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:29.785445 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:29.785533 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:52:29.785580 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260811 18:52:29.786326 20956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81: No bootstrap required, opened a new log
I20260811 18:52:29.786733 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:29.788508 20956 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2f573b3340d4dae97a4eb19daeccc81" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42309 } }
I20260811 18:52:29.788693 20956 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:29.788741 20956 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2f573b3340d4dae97a4eb19daeccc81, State: Initialized, Role: FOLLOWER
I20260811 18:52:29.788878 20956 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2f573b3340d4dae97a4eb19daeccc81" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42309 } }
I20260811 18:52:29.788988 20956 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:29.789058 20956 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:29.789117 20956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:29.789250 20963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:29.788148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40419 (local address 127.9.215.62:42309)
0811 18:52:29.788191 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:29.788193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:29.788310 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:52:29.788367 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:29.788368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:29.788373 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:29.788420 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:29.789040 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:29.789042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:29.789042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:29.789047 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:29.789063 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:29.789067 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:29.789116 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:29.789172 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 18:52:29.789186 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:52:29.789886 20956 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2f573b3340d4dae97a4eb19daeccc81" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42309 } }
I20260811 18:52:29.789999 20956 leader_election.cc:302] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2f573b3340d4dae97a4eb19daeccc81; no voters:
I20260811 18:52:29.790174 20956 leader_election.cc:288] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:29.790220 20975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:29.790402 20975 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [term 1 LEADER]: Becoming Leader. State: Replica: c2f573b3340d4dae97a4eb19daeccc81, State: Running, Role: LEADER
I20260811 18:52:29.790468 20956 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:29.790503 20975 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2f573b3340d4dae97a4eb19daeccc81" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42309 } }
I20260811 18:52:29.790915 20978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2f573b3340d4dae97a4eb19daeccc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2f573b3340d4dae97a4eb19daeccc81" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42309 } } }
I20260811 18:52:29.790927 20986 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2f573b3340d4dae97a4eb19daeccc81. Latest consensus state: current_term: 1 leader_uuid: "c2f573b3340d4dae97a4eb19daeccc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2f573b3340d4dae97a4eb19daeccc81" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 42309 } } }
I20260811 18:52:29.791126 20986 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:29.791113 20978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:29.791730 20998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:29.792263 20998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:29.793880 20998 catalog_manager.cc:1347] Generated new cluster ID: 42759316ab97403a9c9ad8820a4e9574
I20260811 18:52:29.793936 20998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:29.797689 20998 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:29.797873 20998 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:29.807624 20998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2f573b3340d4dae97a4eb19daeccc81: Generated new TSK 0
I20260811 18:52:29.807842 20998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:29.814713 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 20810
2026-08-11T18:52:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:29 dist-test-slave-pww0 krb5kdc[21023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:29 dist-test-slave-pww0 krb5kdc[21023](info): set up 2 sockets
Aug 11 18:52:29 dist-test-slave-pww0 krb5kdc[21023](info): commencing operation
krb5kdc: starting...
W20260811 18:52:31.056605 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:31 dist-test-slave-pww0 krb5kdc[21023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:31.196467 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35727
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:35727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:31.283759 21048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:31.284003 21048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:31.284029 21048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:31.285324 21048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:31.285372 21048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:31.285385 21048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:31.285396 21048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:31.285408 21048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:31.287081 21048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:35727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:35727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:31.287336 21048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:31.287570 21048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:31.292611 21057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:31.292748 21048 server_base.cc:1034] running on GCE node
W20260811 18:52:31.292614 21059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:31.292639 21061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:31.293103 21048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:31.293434 21048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:31.294602 21048 hybrid_clock.cc:648] HybridClock initialized: now 1786474351294581 us; error 33 us; skew 500 ppm
Aug 11 18:52:31 dist-test-slave-pww0 krb5kdc[21023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:31.297326 21048 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:31.297842 21048 webserver.cc:460] Webserver started at http://127.9.215.62:33433/ using document root <none> and password file <none>
I20260811 18:52:31.298069 21048 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:31.298120 21048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:31.298256 21048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:31.299378 21048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "2946f3c4457b47ffaa7e4263264e2b96"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-pww0"
server_key: "06e1d787892ab57318a2557e2c7488e8"
server_key_iv: "d5dac882e81a2c8b5100f3ffb95df397"
server_key_version: "encryptionkey@0"
I20260811 18:52:31.299917 21048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "2946f3c4457b47ffaa7e4263264e2b96"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-pww0"
server_key: "06e1d787892ab57318a2557e2c7488e8"
server_key_iv: "d5dac882e81a2c8b5100f3ffb95df397"
server_key_version: "encryptionkey@0"
I20260811 18:52:31.301453 21048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 18:52:31.302382 21074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:31.302593 21048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:31.302717 21048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "2946f3c4457b47ffaa7e4263264e2b96"
format_stamp: "Formatted at 2026-08-11 18:52:31 on dist-test-slave-pww0"
server_key: "06e1d787892ab57318a2557e2c7488e8"
server_key_iv: "d5dac882e81a2c8b5100f3ffb95df397"
server_key_version: "encryptionkey@0"
I20260811 18:52:31.302790 21048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:31.314817 21048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:31.317226 21048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:31.317391 21048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:31.322299 21048 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:35727
I20260811 18:52:31.322379 21181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:35727 every 8 connection(s)
I20260811 18:52:31.322736 21048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:31.323236 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21048
I20260811 18:52:31.323364 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:31.323609 10076 external_mini_cluster.cc:1428] Setting key 2ccbfdada3009f5932887f54065ea2c2
I20260811 18:52:31.323891 21184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:31.327409 21184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96: Bootstrap starting.
I20260811 18:52:31.328264 21184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:31.328579 21184 log.cc:824] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:31.329373 21184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96: No bootstrap required, opened a new log
Aug 11 18:52:31 dist-test-slave-pww0 krb5kdc[21023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:31.331784 21184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2946f3c4457b47ffaa7e4263264e2b96" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35727 } }
I20260811 18:52:31.331967 21184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:31.332007 21184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2946f3c4457b47ffaa7e4263264e2b96, State: Initialized, Role: FOLLOWER
I20260811 18:52:31.332150 21184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2946f3c4457b47ffaa7e4263264e2b96" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35727 } }
I20260811 18:52:31.332250 21184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:31.332299 21184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:31.332357 21184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:31.333189 21184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2946f3c4457b47ffaa7e4263264e2b96" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35727 } }
I20260811 18:52:31.333317 21184 leader_election.cc:302] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2946f3c4457b47ffaa7e4263264e2b96; no voters:
I20260811 18:52:31.333516 21184 leader_election.cc:288] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:31.333551 21195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:31.333668 21195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [term 1 LEADER]: Becoming Leader. State: Replica: 2946f3c4457b47ffaa7e4263264e2b96, State: Running, Role: LEADER
I20260811 18:52:31.333829 21184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:31.333814 21195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2946f3c4457b47ffaa7e4263264e2b96" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35727 } }
I20260811 18:52:31.334204 21197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2946f3c4457b47ffaa7e4263264e2b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2946f3c4457b47ffaa7e4263264e2b96" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35727 } } }
I20260811 18:52:31.334285 21197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:31.334496 21198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2946f3c4457b47ffaa7e4263264e2b96. Latest consensus state: current_term: 1 leader_uuid: "2946f3c4457b47ffaa7e4263264e2b96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2946f3c4457b47ffaa7e4263264e2b96" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 35727 } } }
I20260811 18:52:31.334635 21198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96 [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:31.335716 21219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:31.335815 21219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:31.335884 21206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:31.336068 21206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:31.336303 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:31.324158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46771 (local address 127.9.215.62:35727)
0811 18:52:31.324356 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:31.324367 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:31.324395 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:52:31.325077 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:31.325087 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:31.325145 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:31.325274 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:31.326101 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.327008 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.328261 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.328779 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.328897 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:31.331686 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:31.331691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:31.331691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:31.331704 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:31.333849 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.334379 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.334382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.334383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.334497 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.334784 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.334785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.334785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.335131 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:31.335804 (+ 673us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:31.335892 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:52:31.336051 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260811 18:52:31.337632 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:31.337738 21206 catalog_manager.cc:1347] Generated new cluster ID: b58bf274b03f4aeebbeaf452a3e72996
I20260811 18:52:31.337790 21206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:31.357426 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:31.349212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45157 (local address 127.9.215.62:35727)
0811 18:52:31.349283 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:31.349286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:31.349409 (+ 123us) server_negotiation.cc:407] Connection header received
0811 18:52:31.349498 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:31.349499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:31.349512 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:31.349574 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:31.350235 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.351060 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.352231 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:31.352583 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:31.352675 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:31.354174 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:31.354177 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:31.354178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:31.354187 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:31.356135 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.356590 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.356591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.356592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.356669 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:31.356852 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:31.356853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:31.356853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:31.357015 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:31.357122 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:31.357220 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:52:31.357320 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:52:31.361565 21206 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:31.362393 21206 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:31.376608 21206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2946f3c4457b47ffaa7e4263264e2b96: Generated new TSK 0
I20260811 18:52:31.376874 21206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:31.380894 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21048
2026-08-11T18:52:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:31 dist-test-slave-pww0 krb5kdc[21239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:31 dist-test-slave-pww0 krb5kdc[21239](info): set up 2 sockets
Aug 11 18:52:31 dist-test-slave-pww0 krb5kdc[21239](info): commencing operation
krb5kdc: starting...
W20260811 18:52:32.668218 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:32 dist-test-slave-pww0 krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:32.803385 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36515
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:36515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:32.888830 21267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:32.889091 21267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:32.889127 21267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:32.890419 21267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:32.890479 21267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:32.890498 21267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:32.890515 21267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:32.890529 21267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:32.892235 21267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:36515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:36515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:32.892532 21267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:32.892777 21267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:32.898182 21275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.898182 21277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:32.898664 21279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:32.898890 21267 server_base.cc:1034] running on GCE node
I20260811 18:52:32.899215 21267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:32.899873 21267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:32.901065 21267 hybrid_clock.cc:648] HybridClock initialized: now 1786474352901044 us; error 35 us; skew 500 ppm
Aug 11 18:52:32 dist-test-slave-pww0 krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474352, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:32.903910 21267 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:32.904428 21267 webserver.cc:460] Webserver started at http://127.9.215.62:42239/ using document root <none> and password file <none>
I20260811 18:52:32.904644 21267 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:32.904698 21267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:32.904834 21267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:32.905970 21267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "92dd67f864404a09ab87d037dd4ab061"
format_stamp: "Formatted at 2026-08-11 18:52:32 on dist-test-slave-pww0"
server_key: "62027d660241f5e4523cebdac055899b"
server_key_iv: "f25b620a642abae789ede7fb3b54f89d"
server_key_version: "encryptionkey@0"
I20260811 18:52:32.906333 21267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "92dd67f864404a09ab87d037dd4ab061"
format_stamp: "Formatted at 2026-08-11 18:52:32 on dist-test-slave-pww0"
server_key: "62027d660241f5e4523cebdac055899b"
server_key_iv: "f25b620a642abae789ede7fb3b54f89d"
server_key_version: "encryptionkey@0"
I20260811 18:52:32.907790 21267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:32.908735 21288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:32.909036 21267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:52:32.909150 21267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "92dd67f864404a09ab87d037dd4ab061"
format_stamp: "Formatted at 2026-08-11 18:52:32 on dist-test-slave-pww0"
server_key: "62027d660241f5e4523cebdac055899b"
server_key_iv: "f25b620a642abae789ede7fb3b54f89d"
server_key_version: "encryptionkey@0"
I20260811 18:52:32.909232 21267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:32.920022 21267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:32.922375 21267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:32.922531 21267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:32.926714 21267 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:36515
I20260811 18:52:32.926757 21399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:36515 every 8 connection(s)
I20260811 18:52:32.927127 21267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:32.928143 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:32.929831 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21267
I20260811 18:52:32.929972 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:32.930168 10076 external_mini_cluster.cc:1428] Setting key 4828574c286bdfce7816c1f0ea7fa3b1
I20260811 18:52:32.930996 21403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061: Bootstrap starting.
I20260811 18:52:32.931823 21403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:32.932169 21403 log.cc:824] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:32.933121 21403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061: No bootstrap required, opened a new log
I20260811 18:52:32.935420 21403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dd67f864404a09ab87d037dd4ab061" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36515 } }
I20260811 18:52:32.935676 21403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:32.935707 21403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92dd67f864404a09ab87d037dd4ab061, State: Initialized, Role: FOLLOWER
I20260811 18:52:32.935873 21403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dd67f864404a09ab87d037dd4ab061" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36515 } }
I20260811 18:52:32.935945 21403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:32.935968 21403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:32.936017 21403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:32.936702 21403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dd67f864404a09ab87d037dd4ab061" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36515 } }
I20260811 18:52:32.936735 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.930802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53219 (local address 127.9.215.62:36515)
0811 18:52:32.930993 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:32.931004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:32.931046 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:52:32.931675 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:32.931684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:32.931754 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:32.931924 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:32.932253 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.933666 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.934516 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.935191 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.935318 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.936093 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:32.936098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:32.936098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:32.936285 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:32.936321 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:32.936333 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:32.936392 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:32.936476 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:52:32.936555 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260811 18:52:32.936823 21403 leader_election.cc:302] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92dd67f864404a09ab87d037dd4ab061; no voters:
I20260811 18:52:32.937024 21403 leader_election.cc:288] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:32.937070 21413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:32.937191 21413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [term 1 LEADER]: Becoming Leader. State: Replica: 92dd67f864404a09ab87d037dd4ab061, State: Running, Role: LEADER
I20260811 18:52:32.937278 21413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dd67f864404a09ab87d037dd4ab061" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36515 } }
I20260811 18:52:32.937306 21403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:32.937757 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:32.939273 21415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92dd67f864404a09ab87d037dd4ab061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dd67f864404a09ab87d037dd4ab061" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36515 } } }
I20260811 18:52:32.939625 21415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:32.939250 21414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92dd67f864404a09ab87d037dd4ab061. Latest consensus state: current_term: 1 leader_uuid: "92dd67f864404a09ab87d037dd4ab061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dd67f864404a09ab87d037dd4ab061" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 36515 } } }
I20260811 18:52:32.939826 21414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061 [sys.catalog]: This master's current role is: LEADER
W20260811 18:52:32.940448 21435 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:32.940518 21435 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:32.940629 21436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:32.940806 21436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:32.942600 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:32.939125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36679 (local address 127.9.215.62:36515)
0811 18:52:32.939166 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:32.939168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:32.939410 (+ 242us) server_negotiation.cc:407] Connection header received
0811 18:52:32.939498 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:32.939499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:32.939512 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:32.939570 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:32.939756 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.940485 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.941303 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:32.941538 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:32.941597 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:32.942380 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:32.942382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:32.942382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:32.942388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:32.942404 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:32.942408 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:32.942452 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:32.942503 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:52:32.942528 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:32.942642 21436 catalog_manager.cc:1347] Generated new cluster ID: 74fc685937b54cdab45b0bb2d05a0a53
I20260811 18:52:32.942694 21436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:32.959251 21436 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:32.959878 21436 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:32.963891 21436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92dd67f864404a09ab87d037dd4ab061: Generated new TSK 0
I20260811 18:52:32.964143 21436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:32.966308 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21267
2026-08-11T18:52:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:33 dist-test-slave-pww0 krb5kdc[21453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:33 dist-test-slave-pww0 krb5kdc[21453](info): set up 2 sockets
Aug 11 18:52:33 dist-test-slave-pww0 krb5kdc[21453](info): commencing operation
krb5kdc: starting...
W20260811 18:52:34.230526 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:34 dist-test-slave-pww0 krb5kdc[21453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:34.376492 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37981
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:34.462311 21481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:34.462540 21481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:34.462565 21481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:34.463838 21481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:34.463893 21481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:34.463907 21481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:34.463918 21481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:34.463929 21481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:34.465719 21481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:34.466042 21481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:34.466311 21481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:34.471500 21491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:34.471505 21494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:34.471558 21503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:34.471784 21481 server_base.cc:1034] running on GCE node
I20260811 18:52:34.472114 21481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:34.472481 21481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:34.473658 21481 hybrid_clock.cc:648] HybridClock initialized: now 1786474354473659 us; error 56 us; skew 500 ppm
Aug 11 18:52:34 dist-test-slave-pww0 krb5kdc[21453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474354, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:34.476511 21481 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:34.477018 21481 webserver.cc:460] Webserver started at http://127.9.215.62:46575/ using document root <none> and password file <none>
I20260811 18:52:34.477244 21481 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:34.477299 21481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:34.477435 21481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:34.478547 21481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "1d1278854f494238a42933c4a1cd6c78"
format_stamp: "Formatted at 2026-08-11 18:52:34 on dist-test-slave-pww0"
server_key: "d84a5309516bf3595e13c111230a257b"
server_key_iv: "0a6bcf0e5c99d69a7ba9521e10a32e57"
server_key_version: "encryptionkey@0"
I20260811 18:52:34.478902 21481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "1d1278854f494238a42933c4a1cd6c78"
format_stamp: "Formatted at 2026-08-11 18:52:34 on dist-test-slave-pww0"
server_key: "d84a5309516bf3595e13c111230a257b"
server_key_iv: "0a6bcf0e5c99d69a7ba9521e10a32e57"
server_key_version: "encryptionkey@0"
I20260811 18:52:34.480358 21481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:34.481139 21519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:34.481284 21481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 18:52:34.481380 21481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "1d1278854f494238a42933c4a1cd6c78"
format_stamp: "Formatted at 2026-08-11 18:52:34 on dist-test-slave-pww0"
server_key: "d84a5309516bf3595e13c111230a257b"
server_key_iv: "0a6bcf0e5c99d69a7ba9521e10a32e57"
server_key_version: "encryptionkey@0"
I20260811 18:52:34.481444 21481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:34.484879 21481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:34.486771 21481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:34.486914 21481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:34.491271 21481 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:37981
I20260811 18:52:34.491305 21624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:37981 every 8 connection(s)
I20260811 18:52:34.491678 21481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:34.492472 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21481
I20260811 18:52:34.492607 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:34.492847 10076 external_mini_cluster.cc:1428] Setting key f26079237b41d9737439eb3b09200f51
I20260811 18:52:34.492908 21625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:34.495384 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:34.493410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37865 (local address 127.9.215.62:37981)
0811 18:52:34.493571 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:34.493579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:34.493605 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:52:34.494060 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:34.494067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:34.494093 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:34.494217 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:34.494953 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:34.494957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:34.494958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:34.494969 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:34.495022 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:34.495037 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:34.495115 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:34.495148 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 18:52:34.495189 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260811 18:52:34.496466 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:34.496572 21625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78: Bootstrap starting.
I20260811 18:52:34.497349 21625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:34.497689 21625 log.cc:824] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:34.498682 21625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78: No bootstrap required, opened a new log
I20260811 18:52:34.500324 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:34.498086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34989 (local address 127.9.215.62:37981)
0811 18:52:34.498122 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:34.498124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:34.498254 (+ 130us) server_negotiation.cc:407] Connection header received
0811 18:52:34.498313 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:34.498314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:34.498318 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:34.498376 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:34.500114 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:34.500116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:34.500116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:52:34.500121 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:34.500139 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:52:34.500144 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:34.500210 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:34.500250 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 18:52:34.500265 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 18:52:34.500989 21625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1278854f494238a42933c4a1cd6c78" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37981 } }
I20260811 18:52:34.501147 21625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:52:34.501169 21625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1278854f494238a42933c4a1cd6c78, State: Initialized, Role: FOLLOWER
I20260811 18:52:34.501298 21625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1278854f494238a42933c4a1cd6c78" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37981 } }
I20260811 18:52:34.501377 21625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:52:34.501403 21625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:52:34.501454 21625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:34.502061 21625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1278854f494238a42933c4a1cd6c78" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37981 } }
I20260811 18:52:34.502162 21625 leader_election.cc:302] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d1278854f494238a42933c4a1cd6c78; no voters:
I20260811 18:52:34.502341 21625 leader_election.cc:288] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:52:34.502410 21645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:34.502576 21645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [term 1 LEADER]: Becoming Leader. State: Replica: 1d1278854f494238a42933c4a1cd6c78, State: Running, Role: LEADER
I20260811 18:52:34.502602 21625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:34.502679 21645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1278854f494238a42933c4a1cd6c78" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37981 } }
I20260811 18:52:34.503105 21646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d1278854f494238a42933c4a1cd6c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1278854f494238a42933c4a1cd6c78" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37981 } } }
I20260811 18:52:34.503214 21646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:34.503378 21649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d1278854f494238a42933c4a1cd6c78. Latest consensus state: current_term: 1 leader_uuid: "1d1278854f494238a42933c4a1cd6c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1278854f494238a42933c4a1cd6c78" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37981 } } }
I20260811 18:52:34.503441 21649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:34.503523 21665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:34.504061 21665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:34.506111 21665 catalog_manager.cc:1347] Generated new cluster ID: 5d96e2b375f14e858940025989bcf99a
I20260811 18:52:34.506160 21665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:34.515370 21665 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:34.515626 21665 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:34.520898 21665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d1278854f494238a42933c4a1cd6c78: Generated new TSK 0
I20260811 18:52:34.521131 21665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:34.524904 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21481
2026-08-11T18:52:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1554 ms)
[----------] 18 tests from AuthTokenIssuingTest (28002 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:34 dist-test-slave-pww0 krb5kdc[21695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:34 dist-test-slave-pww0 krb5kdc[21695](info): set up 2 sockets
Aug 11 18:52:34 dist-test-slave-pww0 krb5kdc[21695](info): commencing operation
krb5kdc: starting...
W20260811 18:52:35.751773 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:35 dist-test-slave-pww0 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:35.887698 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:44557
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:37093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:44557,127.9.215.61:46575,127.9.215.60:43515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:35.982052 21716 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:52:35.982366 21716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:35.982398 21716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:35.982430 21716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:35.983958 21716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:35.984041 21716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:35.984066 21716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:35.984084 21716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:35.984103 21716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:35.986454 21716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:37093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:44557,127.9.215.61:46575,127.9.215.60:43515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:44557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:35.986912 21716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:35.987216 21716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:35.992892 21735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:35.992892 21730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:35.992933 21727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:35.992913 21716 server_base.cc:1034] running on GCE node
I20260811 18:52:35.993449 21716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:35.993837 21716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:35.995051 21716 hybrid_clock.cc:648] HybridClock initialized: now 1786474355995000 us; error 63 us; skew 500 ppm
Aug 11 18:52:35 dist-test-slave-pww0 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:35.998096 21716 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:35.998629 21716 webserver.cc:460] Webserver started at http://127.9.215.62:36137/ using document root <none> and password file <none>
I20260811 18:52:35.998862 21716 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:35.998915 21716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:35.999058 21716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:36.000247 21716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "abb1511a49544886a188bf56b41fe567"
format_stamp: "Formatted at 2026-08-11 18:52:35 on dist-test-slave-pww0"
server_key: "cac4434117ac2677226050cfde677852"
server_key_iv: "26996417eb8acb37432bcb02aabb3fd3"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.000612 21716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "abb1511a49544886a188bf56b41fe567"
format_stamp: "Formatted at 2026-08-11 18:52:35 on dist-test-slave-pww0"
server_key: "cac4434117ac2677226050cfde677852"
server_key_iv: "26996417eb8acb37432bcb02aabb3fd3"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.002106 21716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 18:52:36.003021 21747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:36.003180 21716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:36.003286 21716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "abb1511a49544886a188bf56b41fe567"
format_stamp: "Formatted at 2026-08-11 18:52:35 on dist-test-slave-pww0"
server_key: "cac4434117ac2677226050cfde677852"
server_key_iv: "26996417eb8acb37432bcb02aabb3fd3"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.003362 21716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:36.023531 21716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:36.025849 21716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:36.026024 21716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:36.030457 21716 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:44557
I20260811 18:52:36.030507 21851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:44557 every 8 connection(s)
I20260811 18:52:36.030894 21716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:36.032109 21853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:36.033891 21853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.034390 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21716
I20260811 18:52:36.034497 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:36.034665 10076 external_mini_cluster.cc:1428] Setting key e0ee696b3d860c5d084a7ae5f44d5278
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.215.61@KRBTEST.COM; defaulting to no policy
I20260811 18:52:36.046116 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.034978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44557 (local address 127.0.0.1:42324)
0811 18:52:36.035178 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.035247 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.035838 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.036159 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.036161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.036447 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.036620 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.036624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.037779 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.037780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.038770 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.038774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.039375 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.039377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.039491 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.040068 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.040079 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.041948 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.044398 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.044407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.044411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.044642 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.044902 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.044903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.044903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.045002 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.045616 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.045622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.045684 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.045762 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.045918 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260811 18:52:36.046240 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.034979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.9.215.62:44557)
0811 18:52:36.035231 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.035243 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.035332 (+ 89us) server_negotiation.cc:407] Connection header received
0811 18:52:36.036011 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.036014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.036038 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.036159 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.036787 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.037657 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.038876 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.039243 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.039341 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.042043 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.042058 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.042059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.042065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.044162 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.044725 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.044729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.044729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.044833 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.045085 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.045086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.045087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.045474 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.045564 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.045829 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.046037 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":211,"thread_start_us":118,"threads_started":1}
I20260811 18:52:36.046993 21853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.047413 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.047262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:32944)
0811 18:52:36.047335 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.047390 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:52:36.047610 21750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.61@KRBTEST.COM" created.
I20260811 18:52:36.048081 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.047974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:32950)
0811 18:52:36.048029 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.048059 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 18:52:36.048202 21853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:46575: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab.
Entry for principal kudu/127.9.215.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab.
I20260811 18:52:36.067201 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:46575
--webserver_interface=127.9.215.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab
--principal=kudu/127.9.215.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:37093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:44557,127.9.215.61:46575,127.9.215.60:43515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:36.097466 21853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } attempt: 1
I20260811 18:52:36.098093 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.097914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:32966)
0811 18:52:36.098004 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.098056 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 18:52:36.098487 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.098397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:32978)
0811 18:52:36.098438 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.098465 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 18:52:36.098587 21853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:46575: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
W20260811 18:52:36.156630 21866 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:52:36.156908 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:36.156927 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:36.156950 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:36.158228 21866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:36.158273 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:36.158286 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:36.158298 21866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:36.158309 21866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:36.160056 21866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:37093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:44557,127.9.215.61:46575,127.9.215.60:43515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab
--principal=kudu/127.9.215.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:46575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.215.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:36.160328 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:36.160737 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:36.166347 21873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.166394 21875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.166435 21871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:36.166591 21866 server_base.cc:1034] running on GCE node
I20260811 18:52:36.166743 21853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } attempt: 2
I20260811 18:52:36.166893 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:36.167140 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.167012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:32990)
0811 18:52:36.167065 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.167095 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:52:36.167237 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:36.167527 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.167385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:33000)
0811 18:52:36.167427 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.167497 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260811 18:52:36.167629 21853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:46575: Network error: Client connection negotiation failed: client connection to 127.9.215.61:46575: connect: Connection refused (error 111)
I20260811 18:52:36.168396 21866 hybrid_clock.cc:648] HybridClock initialized: now 1786474356168371 us; error 39 us; skew 500 ppm
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:36.171033 21866 init.cc:377] Logged in from keytab as kudu/127.9.215.61@KRBTEST.COM (short username kudu)
I20260811 18:52:36.171525 21866 webserver.cc:460] Webserver started at http://127.9.215.61:34091/ using document root <none> and password file <none>
I20260811 18:52:36.171741 21866 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:36.171784 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:36.171880 21866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:36.173022 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data/instance:
uuid: "a2cdfa168d0448b3908ecccc9887e964"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-pww0"
server_key: "874a71d5063b16078221484fd1d3aabf"
server_key_iv: "8f2483c1f3668196e9e59f23af8ab900"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.173342 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal/instance:
uuid: "a2cdfa168d0448b3908ecccc9887e964"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-pww0"
server_key: "874a71d5063b16078221484fd1d3aabf"
server_key_iv: "8f2483c1f3668196e9e59f23af8ab900"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.174636 21866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:36.175348 21886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:36.175532 21866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:36.175635 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal
uuid: "a2cdfa168d0448b3908ecccc9887e964"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-pww0"
server_key: "874a71d5063b16078221484fd1d3aabf"
server_key_iv: "8f2483c1f3668196e9e59f23af8ab900"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.175709 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:36.186005 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:36.188271 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:36.188405 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:36.192525 21866 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.61:46575
I20260811 18:52:36.192571 21991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.61:46575 every 8 connection(s)
I20260811 18:52:36.192935 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
I20260811 18:52:36.193995 21992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:36.194461 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 21866
I20260811 18:52:36.194550 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-1/wal/instance
I20260811 18:52:36.194726 10076 external_mini_cluster.cc:1428] Setting key ad605bff2c113c2da80b6265fbf98095
I20260811 18:52:36.195969 21992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.215.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.60@KRBTEST.COM" created.
I20260811 18:52:36.206511 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.196812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.9.215.62:44557)
0811 18:52:36.196879 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.196882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.197086 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:52:36.197606 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.197608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.197620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.197701 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.198366 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.199099 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.200304 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.200623 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.200705 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.203174 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.203176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.203176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.203186 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.204646 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.205136 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.205137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.205138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.205282 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.205619 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.205621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.205621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.205880 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.205968 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.206310 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.206414 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:52:36.206750 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.196773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44557 (local address 127.0.0.1:42326)
0811 18:52:36.196952 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.197000 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.197505 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.197769 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.197773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.198041 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.198265 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.198269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.199260 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.199262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.200208 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.200211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.200739 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.200740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.200865 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.201408 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.201418 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.203049 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.204753 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.204763 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.204765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.205029 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.205383 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.205385 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.205385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.205525 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.205981 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.205990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.206145 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.206293 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.206469 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260811 18:52:36.207410 21992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:36.215550 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.207739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:33012)
0811 18:52:36.207786 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.207798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.207841 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.208070 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.208071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.208226 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.208296 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.208299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.209517 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.209519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.210222 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.210225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.210781 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.210782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.210863 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.211313 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.211318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.212806 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.214526 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.214527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.214528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.214688 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.214866 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.214867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.214867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.214920 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.215194 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.215195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.215235 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.215294 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.215406 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 18:52:36.215620 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.207778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.9.215.61:46575)
0811 18:52:36.207935 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.207940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.207958 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:52:36.207978 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.207979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.207991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.208056 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.208373 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.209404 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.210339 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.210696 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.210769 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.212905 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.212919 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.212920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.212928 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.214444 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.214759 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.214763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.214763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.214813 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.214969 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.214970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.214971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.215134 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.215207 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.215324 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.215489 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260811 18:52:36.216374 21992 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.216751 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.216628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44640)
0811 18:52:36.216665 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.216724 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260811 18:52:36.216928 21888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
I20260811 18:52:36.217392 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.217278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44652)
0811 18:52:36.217331 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.217368 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:52:36.217521 21992 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:43515: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab.
Entry for principal kudu/127.9.215.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab.
I20260811 18:52:36.223127 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:43515
--webserver_interface=127.9.215.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab
--principal=kudu/127.9.215.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:37093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:44557,127.9.215.61:46575,127.9.215.60:43515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:36.258776 21853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } attempt: 3
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:36.265973 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.259267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.9.215.61:46575)
0811 18:52:36.259334 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.259336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.259345 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:36.259373 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.259374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.259389 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.259434 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.259838 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.260347 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.261037 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.261274 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.261331 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.263447 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.263448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.263449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.263481 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.264907 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.265249 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.265250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.265251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.265314 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.265471 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.265472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.265472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.265592 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.265648 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.265786 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.265871 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:52:36.265974 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.259176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:33016)
0811 18:52:36.259240 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.259252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.259300 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.259449 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.259450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.259641 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.259751 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.259754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.260423 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.260425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.260960 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.260962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.261339 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.261341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.261419 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.261938 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.261942 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.263347 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.264985 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.264987 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.264988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.265183 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.265366 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.265367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.265367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.265423 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.265645 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.265646 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.265699 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.265758 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.265870 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 18:52:36.266554 21853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.266726 21992 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } attempt: 1
I20260811 18:52:36.266912 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.266810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44668)
0811 18:52:36.266852 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.266883 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 18:52:36.267078 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.266984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44674)
0811 18:52:36.267028 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.267050 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:52:36.267308 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.267108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44682)
0811 18:52:36.267258 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.267282 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":113,"threads_started":1}
W20260811 18:52:36.267390 21853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:43515: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
I20260811 18:52:36.267417 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.267330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44698)
0811 18:52:36.267369 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.267397 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 18:52:36.267509 21992 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:43515: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
I20260811 18:52:36.298597 21853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } attempt: 1
I20260811 18:52:36.299348 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.298991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44700)
0811 18:52:36.299068 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.299109 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 18:52:36.299785 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.299645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44708)
0811 18:52:36.299708 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.299753 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 18:52:36.299957 21853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:43515: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
W20260811 18:52:36.309875 22002 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:52:36.310112 22002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:36.310128 22002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:36.310148 22002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:36.311352 22002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:36.311407 22002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:36.311421 22002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:36.311432 22002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:36.311443 22002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:36.313165 22002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:37093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:44557,127.9.215.61:46575,127.9.215.60:43515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab
--principal=kudu/127.9.215.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:43515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.215.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:36.313473 22002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:36.313727 22002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:36.319001 22013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.319094 22018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:36.319103 22012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:36.319445 22002 server_base.cc:1034] running on GCE node
I20260811 18:52:36.319705 22002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:36.320009 22002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:36.321171 22002 hybrid_clock.cc:648] HybridClock initialized: now 1786474356321150 us; error 34 us; skew 500 ppm
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:36.324097 22002 init.cc:377] Logged in from keytab as kudu/127.9.215.60@KRBTEST.COM (short username kudu)
I20260811 18:52:36.324596 22002 webserver.cc:460] Webserver started at http://127.9.215.60:34635/ using document root <none> and password file <none>
I20260811 18:52:36.324841 22002 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:36.324892 22002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:36.325028 22002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:36.326176 22002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data/instance:
uuid: "6d3012e36caa4970a75f51381466dae6"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-pww0"
server_key: "0018fc06bae41b0cebb38ec2029f9c93"
server_key_iv: "49a032d74be5040731d301ad0765097a"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.326545 22002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal/instance:
uuid: "6d3012e36caa4970a75f51381466dae6"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-pww0"
server_key: "0018fc06bae41b0cebb38ec2029f9c93"
server_key_iv: "49a032d74be5040731d301ad0765097a"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.328054 22002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:36.328941 22031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:36.329146 22002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:36.329252 22002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal
uuid: "6d3012e36caa4970a75f51381466dae6"
format_stamp: "Formatted at 2026-08-11 18:52:36 on dist-test-slave-pww0"
server_key: "0018fc06bae41b0cebb38ec2029f9c93"
server_key_iv: "49a032d74be5040731d301ad0765097a"
server_key_version: "encryptionkey@0"
I20260811 18:52:36.329324 22002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:36.335644 21992 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } attempt: 2
I20260811 18:52:36.336091 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.335969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44710)
0811 18:52:36.336021 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.336061 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 18:52:36.336388 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.336309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44718)
0811 18:52:36.336342 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.336367 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260811 18:52:36.336464 21992 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:43515: Network error: Client connection negotiation failed: client connection to 127.9.215.60:43515: connect: Connection refused (error 111)
I20260811 18:52:36.350324 22002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:36.352589 22002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:36.352792 22002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:36.357077 22002 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.60:43515
I20260811 18:52:36.357117 22131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.60:43515 every 8 connection(s)
I20260811 18:52:36.357484 22002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
I20260811 18:52:36.358486 22134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:36.358999 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22002
I20260811 18:52:36.359095 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474068101683-10076-0/minicluster-data/master-2/wal/instance
I20260811 18:52:36.359284 10076 external_mini_cluster.cc:1428] Setting key 2a32d62c90ce3126c199a4e828b5b6b9
I20260811 18:52:36.360569 22134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:52:36.369810 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.361313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.9.215.62:44557)
0811 18:52:36.361357 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.361359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.361519 (+ 160us) server_negotiation.cc:407] Connection header received
0811 18:52:36.361592 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.361593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.361604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.361646 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.362162 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.362869 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.364120 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.364408 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.364467 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.366646 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.366648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.366648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.366656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.368406 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.368816 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.368818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.368819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.368881 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.369114 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.369127 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.369127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.369253 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.369303 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.369618 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.369708 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:36.370162 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.361253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:44557 (local address 127.0.0.1:42336)
0811 18:52:36.361414 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.361455 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.361520 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.361664 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.361667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.361942 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.362042 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.362048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.362960 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.362961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.364047 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.364050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.364505 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.364506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.364574 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.365036 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.365047 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.366558 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.368489 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.368496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.368498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.368745 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.368944 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.368945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.368945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.369038 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.369312 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.369319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.369468 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.369586 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.369729 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260811 18:52:36.370576 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.359845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.9.215.60:43515)
0811 18:52:36.360034 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.360045 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.360082 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:52:36.360608 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.360617 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.360682 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.360839 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.361612 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.362835 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.363925 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.364324 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.364432 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.367195 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.367205 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.367205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.367213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.369036 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.369374 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.369377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.369378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.369434 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.369626 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.369627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.369627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.369796 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.369949 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.370175 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.370455 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260811 18:52:36.370751 22134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:36.375118 21853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } attempt: 2
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:36.377682 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.371105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.9.215.61:46575)
0811 18:52:36.371158 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.371161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.371169 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:52:36.371192 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.371193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.371202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.371249 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.371554 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.372129 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.372890 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.373085 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.373128 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.374984 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.374985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.374986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.374994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.376496 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.376971 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.376972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.376973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.377027 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.377206 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.377207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.377207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.377320 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.377377 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.377507 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.377584 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:52:36.377681 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.371039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:46575 (local address 127.0.0.1:33032)
0811 18:52:36.371082 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.371091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.371127 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.371262 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.371262 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.371404 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.371490 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.371492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.372201 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.372202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.372809 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.372811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.373139 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.373140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.373207 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.373617 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.373621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.374891 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.376562 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.376563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.376563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.376730 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.377095 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.377096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.377096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.377155 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.377374 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.377375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.377418 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.377478 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.377579 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:52:36.378568 22134 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:36.378829 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.371396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.9.215.62:44557)
0811 18:52:36.371437 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.371439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.371517 (+ 78us) server_negotiation.cc:407] Connection header received
0811 18:52:36.371585 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.371586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.371594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.371648 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.372059 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.372550 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.373344 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.373615 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.373672 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.376028 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.376029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.376030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.376037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.377629 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.378009 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.378010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.378011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.378067 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.378248 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.378249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.378249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.378381 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.378611 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.378659 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.378758 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:52:36.383136 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.375386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44740)
0811 18:52:36.375442 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.375451 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.375535 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.375666 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.375667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.375896 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.375975 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.375978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.376823 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.376824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.377639 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.377641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.378113 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.378114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.378187 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.378587 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.378591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.380549 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.382075 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.382076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.382076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.382296 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.382485 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.382486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.382486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.382553 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.382805 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.382806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.382855 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.382927 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.383031 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 18:52:36.383139 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.375433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.9.215.60:43515)
0811 18:52:36.375510 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.375512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.375521 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:36.375594 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.375595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.375604 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.375658 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.376061 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.376734 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.377732 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.378028 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.378087 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.380625 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.380627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.380627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.380634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.381997 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.382361 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.382362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.382362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.382423 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.382616 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.382617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.382617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.382742 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.382807 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.382962 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.383039 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:36.385023 21853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567: Bootstrap starting.
I20260811 18:52:36.385855 21853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:36.386255 21853 log.cc:824] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:36.386641 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.378852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44746)
0811 18:52:36.378968 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.378978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.379019 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.379259 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.379260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.379387 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.379423 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.379425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.380919 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.380920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.381580 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.381582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.382100 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.382101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.382142 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.382493 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.382499 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.383912 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.385631 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.385632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.385632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.385819 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.386019 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.386020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.386020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.386071 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.386376 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.386376 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.386412 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.386468 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.386569 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 18:52:36.386955 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.378940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.9.215.60:43515)
0811 18:52:36.379132 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.379134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.379141 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:52:36.379166 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.379166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.379177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.379244 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.379562 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.380834 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.381659 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.381975 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.382046 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.384000 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.384002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.384002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.384009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.385533 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.385904 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.385905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.385905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.385959 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.386124 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.386124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.386124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.386294 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.386354 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.386747 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.386819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
I20260811 18:52:36.387609 21853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567: No bootstrap required, opened a new log
I20260811 18:52:36.388180 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.379374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.9.215.61:46575)
0811 18:52:36.379411 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.379412 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.379419 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:52:36.379477 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.379478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.379487 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.379538 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.380102 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.380834 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.382500 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.382769 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.382826 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.385267 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.385268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.385269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.385276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.386646 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.387012 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.387013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.387014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.387080 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.387307 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.387308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.387308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.387437 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.387975 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.388017 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.388093 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:36.388273 22134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: Bootstrap starting.
I20260811 18:52:36.389021 22134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:36.389307 22134 log.cc:824] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:36.389621 21853 raft_consensus.cc:348] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.389786 21853 raft_consensus.cc:729] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb1511a49544886a188bf56b41fe567, State: Initialized, Role: FOLLOWER
I20260811 18:52:36.389883 21853 consensus_queue.cc:260] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.390095 22134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: No bootstrap required, opened a new log
I20260811 18:52:36.390218 21853 sys_catalog.cc:564] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:36.390177 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.390411 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.391969 22134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.392159 22134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3012e36caa4970a75f51381466dae6, State: Initialized, Role: FOLLOWER
I20260811 18:52:36.392257 22134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
W20260811 18:52:36.392298 22174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:36.392390 22174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:36.392608 22134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:36.393119 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:36.394191 21803 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "abb1511a49544886a188bf56b41fe567"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42344
I20260811 18:52:36.394337 21803 raft_consensus.cc:482] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:52:36.394382 21803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:36.395306 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.395398 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.395541 21803 raft_consensus.cc:504] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.395800 21803 leader_election.cc:288] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [CANDIDATE]: Term 1 election: Requested vote from peers a2cdfa168d0448b3908ecccc9887e964 (127.9.215.61:46575), 6d3012e36caa4970a75f51381466dae6 (127.9.215.60:43515)
I20260811 18:52:36.393446 21716 master_runner.cc:186] Error getting master registration for 127.9.215.61:46575: OK, instance_id { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" instance_seqno: 1786474356191273 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:52:36.395925 21716 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:52:36.396526 22195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:36.396587 22195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:36.396616 22079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "abb1511a49544886a188bf56b41fe567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6d3012e36caa4970a75f51381466dae6"
I20260811 18:52:36.396727 22079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:36.397455 22079 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb1511a49544886a188bf56b41fe567 in term 1.
I20260811 18:52:36.397742 21750 leader_election.cc:302] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d3012e36caa4970a75f51381466dae6, abb1511a49544886a188bf56b41fe567; no voters:
I20260811 18:52:36.397890 22151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:36.398066 22151 raft_consensus.cc:686] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [term 1 LEADER]: Becoming Leader. State: Replica: abb1511a49544886a188bf56b41fe567, State: Running, Role: LEADER
I20260811 18:52:36.398154 22151 consensus_queue.cc:237] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.398578 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb1511a49544886a188bf56b41fe567. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.398675 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:36.399055 22210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:36.399317 22210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:36.401171 22079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [term 1 FOLLOWER]: Refusing update from remote peer abb1511a49544886a188bf56b41fe567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:36.398893 22002 master_runner.cc:186] Error getting master registration for 127.9.215.61:46575: OK, instance_id { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" instance_seqno: 1786474356191273 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:52:36.401286 22002 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:52:36.401515 22197 consensus_queue.cc:1035] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:36.402849 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb1511a49544886a188bf56b41fe567. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.402989 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.404153 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.404253 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:36.404474 22177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.404570 22177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.404675 22210 catalog_manager.cc:1347] Generated new cluster ID: 99cba3571b73484aaf48efc3694a8a30
I20260811 18:52:36.404723 22210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:36.404862 22225 mvcc.cc:205] Tried to move back new op lower bound from 7317398963816706048 to 7317398963807518720. Current Snapshot: MvccSnapshot[applied={T|T < 7317398963816706048}]
I20260811 18:52:36.416447 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.398728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.9.215.62:44557)
0811 18:52:36.398783 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.398785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.407638 (+ 8853us) server_negotiation.cc:407] Connection header received
0811 18:52:36.407748 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.407749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.407761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.407814 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.408783 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.409505 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.410655 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.410949 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.411005 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.413150 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.413152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.413152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.413161 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.414842 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.415405 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.415406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.415406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.415509 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.415778 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.415778 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.415779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.415912 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.415978 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.416267 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.416347 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 18:52:36.424086 22210 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:36.424371 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.399063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.9.215.61:46575)
0811 18:52:36.399104 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.399106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.407816 (+ 8710us) server_negotiation.cc:407] Connection header received
0811 18:52:36.407868 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.407869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.407881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.407948 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.416130 (+ 8182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.416846 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.417867 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.418160 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.418218 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.419856 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.419859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.419859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.419868 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.421563 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.422233 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.422235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.422235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.422307 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.423161 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.423174 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.423175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.423335 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.423430 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.424167 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.424272 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 18:52:36.425066 22210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:36.426712 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.399392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.9.215.60:43515)
0811 18:52:36.399425 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.399427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.417670 (+ 18243us) server_negotiation.cc:407] Connection header received
0811 18:52:36.417796 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.417797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.417812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.417837 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.418794 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.419592 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.420987 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.421329 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.421374 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.423173 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.423175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.423175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.423183 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.425113 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.425676 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.425677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.425678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.425738 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.425937 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.425938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.425938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.426094 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.426204 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.426520 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.426617 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 18:52:36.427582 21992 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } attempt: 3
Aug 11 18:52:36 dist-test-slave-pww0 krb5kdc[21695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474356, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:52:36.432729 22210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567: Generated new TSK 0
I20260811 18:52:36.433254 22210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:36.441148 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.427844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:43515 (local address 127.0.0.1:44776)
0811 18:52:36.427926 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:36.427936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:36.427972 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:36.428199 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:36.428200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:36.428375 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:36.428427 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.428429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.429327 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.429328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.430172 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:36.430174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.430682 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.430683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:36.430746 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.431222 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:36.431227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:36.432822 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:36.436296 (+ 3474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.436298 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.436299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.436541 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.440286 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:36.440288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:36.440288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:36.440421 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:36.440756 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:36.440758 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:36.440815 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 18:52:36.440879 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:52:36.441025 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 18:52:36.441695 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.428056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.9.215.60:43515)
0811 18:52:36.428095 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.428096 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.428116 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:52:36.428133 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.428134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.428143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.428192 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.428501 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.429259 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.430315 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.430627 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.430684 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.432913 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:36.432915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:36.432915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:36.432924 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:36.436206 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.439931 (+ 3725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.439934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.439935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.440054 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:36.440497 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:36.440498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:36.440499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:36.440677 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:36.440739 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.441361 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.441548 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:36.443475 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: Bootstrap starting.
I20260811 18:52:36.444329 21992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:36.444657 21992 log.cc:824] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:36.445840 21992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: No bootstrap required, opened a new log
I20260811 18:52:36.448058 21992 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.448246 21992 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cdfa168d0448b3908ecccc9887e964, State: Initialized, Role: FOLLOWER
I20260811 18:52:36.448345 21992 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } }
I20260811 18:52:36.448709 21992 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:36.448827 21938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:36.448774 21941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "abb1511a49544886a188bf56b41fe567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a2cdfa168d0448b3908ecccc9887e964"
I20260811 18:52:36.449800 21938 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [term 1 FOLLOWER]: Refusing update from remote peer abb1511a49544886a188bf56b41fe567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:36.450590 21941 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb1511a49544886a188bf56b41fe567 in term 1.
I20260811 18:52:36.451180 22151 consensus_queue.cc:1035] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:36.453886 22257 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.453965 22257 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.454008 22257 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb1511a49544886a188bf56b41fe567. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.454042 22257 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.454066 22257 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.454105 22257 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:36.457392 22256 catalog_manager.cc:1259] Loaded cluster ID: 99cba3571b73484aaf48efc3694a8a30
I20260811 18:52:36.457464 22256 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: loading cluster ID for follower catalog manager: success
I20260811 18:52:36.458653 22256 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: acquiring CA information for follower catalog manager: success
I20260811 18:52:36.459040 22256 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2cdfa168d0448b3908ecccc9887e964: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:52:36.473434 21859 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.471449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.9.215.62:44557)
0811 18:52:36.471555 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.471557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.472613 (+ 1056us) server_negotiation.cc:407] Connection header received
0811 18:52:36.472724 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.472726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.472738 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.472769 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.473330 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42368: BlockingRecv error: recv got EOF from 127.0.0.1:42368 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 18:52:36.474299 21998 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.471945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.9.215.61:46575)
0811 18:52:36.471989 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.471991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.473722 (+ 1731us) server_negotiation.cc:407] Connection header received
0811 18:52:36.473825 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.473826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.473838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.473868 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.474231 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33054: BlockingRecv error: recv got EOF from 127.0.0.1:33054 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 18:52:36.475060 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.472314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.9.215.60:43515)
0811 18:52:36.472363 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.472365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.474548 (+ 2183us) server_negotiation.cc:407] Connection header received
0811 18:52:36.474645 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.474647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.474658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.474686 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.474996 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44784: BlockingRecv error: recv got EOF from 127.0.0.1:44784 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 18:52:36.480720 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.476934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.9.215.61:46575)
0811 18:52:36.476990 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.476992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.477138 (+ 146us) server_negotiation.cc:407] Connection header received
0811 18:52:36.477257 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.477259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.477314 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.477363 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.477574 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.478379 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.479697 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.479990 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.480029 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.480289 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.480445 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.480500 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.480612 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.480635 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 18:52:36.484411 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.483509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.9.215.60:43515)
0811 18:52:36.483558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.483560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.483751 (+ 191us) server_negotiation.cc:407] Connection header received
0811 18:52:36.483869 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.483870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.483894 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.483924 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.484346 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44792: BlockingRecv error: recv got EOF from 127.0.0.1:44792 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:52:36.490370 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.486798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.9.215.61:46575)
0811 18:52:36.486839 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.486841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.487092 (+ 251us) server_negotiation.cc:407] Connection header received
0811 18:52:36.487211 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.487213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.487224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.487252 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.487436 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.488244 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.489435 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.489747 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.489798 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.490051 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.490089 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.490125 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.490227 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.490253 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 18:52:36.494192 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.493220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.9.215.60:43515)
0811 18:52:36.493272 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.493274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.493578 (+ 304us) server_negotiation.cc:407] Connection header received
0811 18:52:36.493685 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.493686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.493699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.493728 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.494123 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44798: BlockingRecv error: recv got EOF from 127.0.0.1:44798 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:52:36.501175 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.497594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.9.215.61:46575)
0811 18:52:36.497641 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.497643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.497921 (+ 278us) server_negotiation.cc:407] Connection header received
0811 18:52:36.498025 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.498027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.498038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.498066 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.498270 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.499045 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.500320 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.500636 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.500678 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.500886 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.500925 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.500951 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.501050 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.501072 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 18:52:36.504711 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.503815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.9.215.60:43515)
0811 18:52:36.503859 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.503861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.504139 (+ 278us) server_negotiation.cc:407] Connection header received
0811 18:52:36.504240 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.504241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.504253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.504280 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.504650 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44814: BlockingRecv error: recv got EOF from 127.0.0.1:44814 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:36.513554 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.510027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.9.215.61:46575)
0811 18:52:36.510071 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.510073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.510374 (+ 301us) server_negotiation.cc:407] Connection header received
0811 18:52:36.510475 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.510476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.510486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.510514 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.510691 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.511540 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.512724 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.513031 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.513071 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.513287 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.513325 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.513350 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.513446 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.513467 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 18:52:36.517208 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.516290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.9.215.60:43515)
0811 18:52:36.516339 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.516341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.516617 (+ 276us) server_negotiation.cc:407] Connection header received
0811 18:52:36.516720 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.516721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.516733 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.516763 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.517145 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44830: BlockingRecv error: recv got EOF from 127.0.0.1:44830 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:36.530073 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.526652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.9.215.61:46575)
0811 18:52:36.526700 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.526702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.526996 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:52:36.527118 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.527119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.527130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.527161 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.527352 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.528177 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.529315 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.529614 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.529651 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.529827 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.529861 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.529883 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.529971 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.529990 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 18:52:36.533658 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.532757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.9.215.60:43515)
0811 18:52:36.532801 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.532802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.533049 (+ 247us) server_negotiation.cc:407] Connection header received
0811 18:52:36.533185 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.533186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.533199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.533228 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.533598 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44836: BlockingRecv error: recv got EOF from 127.0.0.1:44836 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:36.554809 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.551223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.9.215.61:46575)
0811 18:52:36.551274 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.551276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.551629 (+ 353us) server_negotiation.cc:407] Connection header received
0811 18:52:36.551745 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.551746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.551757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.551788 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.551999 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.552754 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.553928 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.554226 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.554277 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.554510 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.554550 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.554579 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.554700 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.554725 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 18:52:36.558712 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.557770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.9.215.60:43515)
0811 18:52:36.557820 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.557822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.558117 (+ 295us) server_negotiation.cc:407] Connection header received
0811 18:52:36.558222 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.558224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.558236 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.558268 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.558650 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44852: BlockingRecv error: recv got EOF from 127.0.0.1:44852 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:52:36.595953 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.592309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.9.215.61:46575)
0811 18:52:36.592359 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.592361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.592693 (+ 332us) server_negotiation.cc:407] Connection header received
0811 18:52:36.592809 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.592811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.592822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.592851 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.593067 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.593817 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.595042 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.595345 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.595392 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.595678 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.595716 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.595744 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.595845 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.595868 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 18:52:36.599545 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.598726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.9.215.60:43515)
0811 18:52:36.598771 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.598773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.598956 (+ 183us) server_negotiation.cc:407] Connection header received
0811 18:52:36.599045 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.599045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.599056 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.599083 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.599448 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44854: BlockingRecv error: recv got EOF from 127.0.0.1:44854 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:36.667287 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.664975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.9.215.61:46575)
0811 18:52:36.665089 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.665091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.665104 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:52:36.665184 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.665185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.665195 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.665250 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.665402 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.665911 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.666749 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.666942 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.667001 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.667103 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.667133 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.667166 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.667207 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.667223 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 18:52:36.669665 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.669054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.9.215.60:43515)
0811 18:52:36.669125 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.669127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.669139 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:52:36.669200 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.669201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.669212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.669265 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.669593 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44856: BlockingRecv error: recv got EOF from 127.0.0.1:44856 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:52:36.801391 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:36.798774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.9.215.61:46575)
0811 18:52:36.798863 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.798865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.798917 (+ 52us) server_negotiation.cc:407] Connection header received
0811 18:52:36.798968 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.798969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.798979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.799036 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:36.799156 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.799735 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.800575 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:36.800775 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:36.800840 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:36.801057 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:36.801098 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:36.801145 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:36.801295 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 18:52:36.801320 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 18:52:36.804733 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:36.804018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44870 (local address 127.9.215.60:43515)
0811 18:52:36.804121 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:36.804123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:36.804137 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:52:36.804201 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:36.804202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:36.804212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:36.804271 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:36.804657 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44870: BlockingRecv error: recv got EOF from 127.0.0.1:44870 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:52:36.883641 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "abb1511a49544886a188bf56b41fe567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb1511a49544886a188bf56b41fe567" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 44557 } } peers { permanent_uuid: "a2cdfa168d0448b3908ecccc9887e964" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 46575 } } peers { permanent_uuid: "6d3012e36caa4970a75f51381466dae6" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 43515 } } }
I20260811 18:52:36.883800 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb1511a49544886a188bf56b41fe567 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:37.064723 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.062020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33118 (local address 127.9.215.61:46575)
0811 18:52:37.062106 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.062108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.062119 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:52:37.062151 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.062152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.062161 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.062228 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:37.062481 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.062973 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.063850 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.064049 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.064109 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.064406 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:37.064468 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:37.064522 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.064613 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.064635 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 18:52:37.066996 22145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:37.066486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.9.215.60:43515)
0811 18:52:37.066551 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.066553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.066563 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:37.066604 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.066604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.066615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.066660 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:37.066948 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44878: BlockingRecv error: recv got EOF from 127.0.0.1:44878 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:52:37.397053 22195 catalog_manager.cc:1259] Loaded cluster ID: 99cba3571b73484aaf48efc3694a8a30
I20260811 18:52:37.397157 22195 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: loading cluster ID for follower catalog manager: success
I20260811 18:52:37.397987 22195 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: acquiring CA information for follower catalog manager: success
I20260811 18:52:37.398303 22195 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6d3012e36caa4970a75f51381466dae6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:52:37.582535 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.580226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.9.215.61:46575)
0811 18:52:37.580365 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.580367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.580376 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:37.580404 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.580406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.580417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.580471 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:37.580606 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.581111 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.581881 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.582084 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.582139 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.582282 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:37.582318 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:37.582360 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.582414 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.582441 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260811 18:52:37.587257 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.584472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44886 (local address 127.9.215.60:43515)
0811 18:52:37.584613 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.584615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.584623 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:52:37.584649 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.584650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.584705 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.584784 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:37.584981 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.585494 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.586310 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.586558 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.586623 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.586851 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:37.586981 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:37.587065 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.587137 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.587162 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260811 18:52:37.592247 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.589335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.9.215.60:43515)
0811 18:52:37.589384 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.589386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.589516 (+ 130us) server_negotiation.cc:407] Connection header received
0811 18:52:37.589552 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.589553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.589564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.589629 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:37.589845 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.590615 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.591447 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.591754 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.591846 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.591981 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:37.592031 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:37.592076 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.592139 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.592162 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:37.592433 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.589157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.9.215.61:46575)
0811 18:52:37.589216 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.589218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.589364 (+ 146us) server_negotiation.cc:407] Connection header received
0811 18:52:37.589475 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.589476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.589489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.589516 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:37.589705 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.590473 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.591574 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.591857 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.591911 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.592119 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:37.592153 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:37.592182 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.592322 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.592351 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 18:52:37.592437 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:37.589193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.9.215.62:44557)
0811 18:52:37.589311 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:37.589312 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:37.589434 (+ 122us) server_negotiation.cc:407] Connection header received
0811 18:52:37.589517 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:37.589518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:37.589578 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:37.589655 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:37.589804 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.590350 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.591571 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:37.591775 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:37.591815 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:37.592051 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:37.592178 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:37.592237 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:37.592329 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 18:52:37.592356 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260811 18:52:37.594501 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21716
I20260811 18:52:37.611310 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 21866
I20260811 18:52:37.619872 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22002
2026-08-11T18:52:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3103 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:52:37 dist-test-slave-pww0 krb5kdc[22509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:52:37 dist-test-slave-pww0 krb5kdc[22509](info): set up 2 sockets
Aug 11 18:52:37 dist-test-slave-pww0 krb5kdc[22509](info): commencing operation
krb5kdc: starting...
W20260811 18:52:38.872572 10076 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/test-user.keytab.
Aug 11 18:52:38 dist-test-slave-pww0 krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.215.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
Entry for principal kudu/127.9.215.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab.
I20260811 18:52:39.008015 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37739
--webserver_interface=127.9.215.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37739,127.9.215.61:34073,127.9.215.60:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:52:39.094798 22536 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:52:39.095044 22536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:39.095069 22536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:39.095099 22536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:39.096410 22536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:39.096472 22536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:39.096493 22536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:39.096508 22536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:39.096524 22536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:39.098228 22536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37739,127.9.215.61:34073,127.9.215.60:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.62.keytab
--principal=kudu/127.9.215.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.62:37739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.215.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:39.098522 22536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:39.098767 22536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:39.104038 22555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:39.104031 22546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:39.104095 22536 server_base.cc:1034] running on GCE node
W20260811 18:52:39.104049 22549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:39.104532 22536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:39.104851 22536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:39.106024 22536 hybrid_clock.cc:648] HybridClock initialized: now 1786474359105990 us; error 50 us; skew 500 ppm
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:39.109004 22536 init.cc:377] Logged in from keytab as kudu/127.9.215.62@KRBTEST.COM (short username kudu)
I20260811 18:52:39.109486 22536 webserver.cc:460] Webserver started at http://127.9.215.62:33657/ using document root <none> and password file <none>
I20260811 18:52:39.109706 22536 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:39.109756 22536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:39.109889 22536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:39.111045 22536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data/instance:
uuid: "4e83be23305e409d81a0a865cd4fd3b3"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "867175a02e70a7d88d609ee35fca8718"
server_key_iv: "96d70839f0428345fda66ec8c9c82d46"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.111385 22536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal/instance:
uuid: "4e83be23305e409d81a0a865cd4fd3b3"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "867175a02e70a7d88d609ee35fca8718"
server_key_iv: "96d70839f0428345fda66ec8c9c82d46"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.112774 22536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:39.113636 22568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:39.113893 22536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:52:39.114002 22536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal
uuid: "4e83be23305e409d81a0a865cd4fd3b3"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "867175a02e70a7d88d609ee35fca8718"
server_key_iv: "96d70839f0428345fda66ec8c9c82d46"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.114075 22536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:39.131788 22536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:39.134158 22536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:39.134312 22536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:39.138602 22536 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.62:37739
I20260811 18:52:39.138636 22664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.62:37739 every 8 connection(s)
I20260811 18:52:39.139019 22536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/data/info.pb
I20260811 18:52:39.140167 22665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:39.141841 22665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.144354 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22536
I20260811 18:52:39.144448 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-0/wal/instance
I20260811 18:52:39.144630 10076 external_mini_cluster.cc:1428] Setting key ac5b5f8a045a8df2a74ab4c975e0ad32
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:39.154233 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.142568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37739 (local address 127.0.0.1:53488)
0811 18:52:39.142777 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.142816 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.143307 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.143651 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.143654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.143940 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.144109 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.144114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.145275 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.145276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.146293 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.146297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.147117 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.147119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.147189 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.147816 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.147825 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.149482 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.151975 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.151981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.151984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.152304 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.152805 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.152806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.152806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.152921 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.153480 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.153489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.153595 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.153722 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.153942 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260811 18:52:39.154806 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.142764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.9.215.62:37739)
0811 18:52:39.142896 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.142901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.142936 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:52:39.143514 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.143517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.143540 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.143625 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.144201 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.145169 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.146419 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.146973 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.147097 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.149624 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.149634 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.149634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.149642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.151867 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.152486 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.152491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.152492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.152718 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.153029 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.153031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.153031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.153394 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.153484 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.154583 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.154700 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260811 18:52:39.155618 22665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.156088 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.155926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35596)
0811 18:52:39.155986 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.156055 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
WARNING: no policy specified for kudu/127.9.215.61@KRBTEST.COM; defaulting to no policy
W20260811 18:52:39.156348 22573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
I20260811 18:52:39.156878 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.156766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35610)
0811 18:52:39.156804 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.156850 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260811 18:52:39.157003 22665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:34073: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab.
Entry for principal kudu/127.9.215.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab.
I20260811 18:52:39.179318 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:34073
--webserver_interface=127.9.215.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab
--principal=kudu/127.9.215.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37739,127.9.215.61:34073,127.9.215.60:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:39.206287 22665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } attempt: 1
I20260811 18:52:39.206851 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.206677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35624)
0811 18:52:39.206750 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.206807 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 18:52:39.207226 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.207123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35640)
0811 18:52:39.207174 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.207202 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:52:39.207329 22665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:34073: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
W20260811 18:52:39.265336 22680 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:52:39.265574 22680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:39.265591 22680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:39.265614 22680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:39.266798 22680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:39.266834 22680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:39.266847 22680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:39.266858 22680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:39.266870 22680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:39.268437 22680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37739,127.9.215.61:34073,127.9.215.60:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.61.keytab
--principal=kudu/127.9.215.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.61:34073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.215.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:39.268680 22680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:39.268877 22680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:52:39.274183 22695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:39.274168 22698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:39.274226 22680 server_base.cc:1034] running on GCE node
W20260811 18:52:39.274171 22691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:39.274667 22680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:39.275041 22680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:39.275492 22665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } attempt: 2
I20260811 18:52:39.275899 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.275776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35652)
0811 18:52:39.275830 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.275871 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 18:52:39.276194 22680 hybrid_clock.cc:648] HybridClock initialized: now 1786474359276170 us; error 46 us; skew 500 ppm
I20260811 18:52:39.276330 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.276213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35654)
0811 18:52:39.276262 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.276296 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 18:52:39.276428 22665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.61:34073: Network error: Client connection negotiation failed: client connection to 127.9.215.61:34073: connect: Connection refused (error 111)
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:39.284751 22680 init.cc:377] Logged in from keytab as kudu/127.9.215.61@KRBTEST.COM (short username kudu)
I20260811 18:52:39.285307 22680 webserver.cc:460] Webserver started at http://127.9.215.61:34793/ using document root <none> and password file <none>
I20260811 18:52:39.285512 22680 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:39.285555 22680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:39.285655 22680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:39.286734 22680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data/instance:
uuid: "6750d0a9e39742aab163fdc117751dc8"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "8084320aa17877b8494cee576fe51efa"
server_key_iv: "15e08be8fd10f7f38463aebf56b614af"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.287040 22680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal/instance:
uuid: "6750d0a9e39742aab163fdc117751dc8"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "8084320aa17877b8494cee576fe51efa"
server_key_iv: "15e08be8fd10f7f38463aebf56b614af"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.288425 22680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 18:52:39.289140 22712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:39.289301 22680 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 18:52:39.289391 22680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal
uuid: "6750d0a9e39742aab163fdc117751dc8"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "8084320aa17877b8494cee576fe51efa"
server_key_iv: "15e08be8fd10f7f38463aebf56b614af"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.289455 22680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:39.300783 22680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:39.303049 22680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:39.303236 22680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:39.307296 22680 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.61:34073
I20260811 18:52:39.307344 22812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.61:34073 every 8 connection(s)
I20260811 18:52:39.307745 22680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/data/info.pb
I20260811 18:52:39.308760 22815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:39.310424 22815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.315650 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22680
I20260811 18:52:39.315778 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-1/wal/instance
I20260811 18:52:39.316001 10076 external_mini_cluster.cc:1428] Setting key aaae18208b525d926366c47d45cf34d0
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:39.320894 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.311209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53504 (local address 127.9.215.62:37739)
0811 18:52:39.311278 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.311280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.311537 (+ 257us) server_negotiation.cc:407] Connection header received
0811 18:52:39.312092 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.312093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.312106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.312146 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.312827 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.313334 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.314876 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.315107 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.315174 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.317812 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.317814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.317814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.317822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.319124 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.319681 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.319683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.319683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.319753 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.320091 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.320093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.320093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.320252 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.320318 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.320721 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.320812 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:52:39.321107 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.311164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37739 (local address 127.0.0.1:53504)
0811 18:52:39.311382 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.311434 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.311995 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.312221 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.312225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.312492 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.312739 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.312744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.313434 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.313435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.314766 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.314771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.315246 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.315247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.315425 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.316125 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.316137 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.317689 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.319217 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.319229 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.319232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.319587 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.319825 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.319827 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.319827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.319983 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.320321 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.320330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.320516 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.320627 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.320843 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260811 18:52:39.321923 22815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.215.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.215.60@KRBTEST.COM" created.
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:39.332185 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.322198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35664)
0811 18:52:39.322256 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.322270 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.322319 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.322546 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.322547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.322711 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.322785 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.322788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.323795 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.323796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.324586 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.324589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.325117 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.325118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.325208 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.325616 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.325621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.329068 (+ 3447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.330949 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.330950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.330951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.331171 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.331424 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.331425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.331425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.331513 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.331823 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.331824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.331867 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.331932 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.332056 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 18:52:39.332248 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.322239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.9.215.61:34073)
0811 18:52:39.322405 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.322411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.322431 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:52:39.322451 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.322453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.322465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.322528 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.322900 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.323706 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.324694 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.325035 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.325095 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.329171 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.329186 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.329186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.329195 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.330845 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.331281 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.331288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.331288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.331353 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.331576 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.331577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.331577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.331750 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.331835 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.331970 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.332125 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260811 18:52:39.332871 22815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.333274 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.333148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60448)
0811 18:52:39.333193 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.333247 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 18:52:39.333437 22714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
I20260811 18:52:39.333905 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.333834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60462)
0811 18:52:39.333862 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.333887 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260811 18:52:39.334012 22815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:34891: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.215.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab.
Entry for principal kudu/127.9.215.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab.
I20260811 18:52:39.346165 10076 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
/tmp/dist-test-taskdX0PXQ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:34891
--webserver_interface=127.9.215.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab
--principal=kudu/127.9.215.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.215.20:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.215.62:37739,127.9.215.61:34073,127.9.215.60:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:52:39.367558 22665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } attempt: 3
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:39.374660 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.368060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.9.215.61:34073)
0811 18:52:39.368128 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.368130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.368139 (+ 9us) server_negotiation.cc:407] Connection header received
0811 18:52:39.368165 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.368167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.368177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.368224 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.368558 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.369058 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.369796 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.370078 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.370149 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.372278 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.372280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.372280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.372288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.373579 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.373972 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.373973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.373974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.374033 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.374205 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.374205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.374206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.374316 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.374365 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.374511 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.374580 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:52:39.374701 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.367973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35666)
0811 18:52:39.368030 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.368043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.368086 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.368235 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.368235 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.368403 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.368475 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.368477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.369152 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.369153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.369712 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.369714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.370149 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.370150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.370230 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.370777 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.370783 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.372160 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.373662 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.373663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.373664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.373896 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.374088 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.374089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.374090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.374153 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.374368 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.374368 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.374426 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.374493 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.374600 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 18:52:39.375231 22665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.375645 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.375535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60476)
0811 18:52:39.375580 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.375618 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 18:52:39.375999 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.375905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60480)
0811 18:52:39.375946 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.375974 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 18:52:39.376093 22665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:34891: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
I20260811 18:52:39.383287 22815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } attempt: 1
I20260811 18:52:39.383872 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.383685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60488)
0811 18:52:39.383787 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.383834 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 18:52:39.384281 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.384144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60494)
0811 18:52:39.384208 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.384254 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:52:39.384404 22815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:34891: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
I20260811 18:52:39.407279 22665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } attempt: 1
I20260811 18:52:39.407878 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.407694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60496)
0811 18:52:39.407789 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.407841 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:52:39.408300 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.408189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60504)
0811 18:52:39.408247 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.408274 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 18:52:39.408406 22665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:34891: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
W20260811 18:52:39.433583 22829 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:52:39.433852 22829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:52:39.433878 22829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:52:39.433908 22829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:52:39.435177 22829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:52:39.435247 22829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:52:39.435266 22829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:52:39.435283 22829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:52:39.435298 22829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:52:39.437150 22829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.215.20:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.215.62:37739,127.9.215.61:34073,127.9.215.60:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/krb5kdc/kudu_127.9.215.60.keytab
--principal=kudu/127.9.215.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.215.60:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.215.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 17:59:43 UTC on ccycloud.quasar-xuakos.root.comops.site
build id 4581133
I20260811 18:52:39.437484 22829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:52:39.437729 22829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:52:39.442926 22829 server_base.cc:1034] running on GCE node
W20260811 18:52:39.442883 22849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:39.442903 22846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:52:39.442903 22847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:52:39.443500 22829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:52:39.443817 22829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:52:39.444968 22829 hybrid_clock.cc:648] HybridClock initialized: now 1786474359444953 us; error 33 us; skew 500 ppm
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:52:39.447546 22829 init.cc:377] Logged in from keytab as kudu/127.9.215.60@KRBTEST.COM (short username kudu)
I20260811 18:52:39.447989 22829 webserver.cc:460] Webserver started at http://127.9.215.60:39727/ using document root <none> and password file <none>
I20260811 18:52:39.448202 22829 fs_manager.cc:359] Metadata directory not provided
I20260811 18:52:39.448246 22829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:52:39.448347 22829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:52:39.449426 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data/instance:
uuid: "3fd77eb92cd6485bbc1ec99115872f44"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "fdf1ddbebb917314040af70c7e4ff037"
server_key_iv: "91e09490610585f0213a7ac1089f8a49"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.449752 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal/instance:
uuid: "3fd77eb92cd6485bbc1ec99115872f44"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "fdf1ddbebb917314040af70c7e4ff037"
server_key_iv: "91e09490610585f0213a7ac1089f8a49"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.451068 22829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 18:52:39.451867 22860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:52:39.452062 22829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:52:39.452157 22829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data,/tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal
uuid: "3fd77eb92cd6485bbc1ec99115872f44"
format_stamp: "Formatted at 2026-08-11 18:52:39 on dist-test-slave-pww0"
server_key: "fdf1ddbebb917314040af70c7e4ff037"
server_key_iv: "91e09490610585f0213a7ac1089f8a49"
server_key_version: "encryptionkey@0"
I20260811 18:52:39.452217 22829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:52:39.452579 22815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } attempt: 2
I20260811 18:52:39.453054 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.452935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60508)
0811 18:52:39.452988 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.453026 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 18:52:39.453373 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.453295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60520)
0811 18:52:39.453332 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.453356 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 18:52:39.453469 22815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.215.60:34891: Network error: Client connection negotiation failed: client connection to 127.9.215.60:34891: connect: Connection refused (error 111)
I20260811 18:52:39.470291 22829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:52:39.472358 22829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:52:39.472481 22829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:52:39.476491 22829 rpc_server.cc:307] RPC server started. Bound to: 127.9.215.60:34891
I20260811 18:52:39.476554 22953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.215.60:34891 every 8 connection(s)
I20260811 18:52:39.476928 22829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/data/info.pb
I20260811 18:52:39.478118 22954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:52:39.479988 22954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.482417 10076 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu as pid 22829
I20260811 18:52:39.482534 10076 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdX0PXQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474068101683-10076-0/minicluster-data/master-2/wal/instance
I20260811 18:52:39.482749 10076 external_mini_cluster.cc:1428] Setting key d7dbf79491bb593e2e20dd265465da1d
I20260811 18:52:39.483511 22665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } attempt: 2
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.62@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:52:39.491784 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.480815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.9.215.62:37739)
0811 18:52:39.480889 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.480892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.481172 (+ 280us) server_negotiation.cc:407] Connection header received
0811 18:52:39.481666 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.481668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.481680 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.481748 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.482419 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.483081 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.484450 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.484726 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.484795 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.488184 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.488185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.488186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.488193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.489725 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.490251 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.490252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.490253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.490329 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.490867 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.490869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.490869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.491116 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.491207 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.491597 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.491683 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:52:39.492005 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.480780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.62:37739 (local address 127.0.0.1:53518)
0811 18:52:39.481005 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.481052 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.481560 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.481811 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.481815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.482089 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.482308 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.482313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.483178 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.483179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.484374 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.484376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.484832 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.484833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.484969 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.486373 (+ 1404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.486384 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.488071 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.489792 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.489800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.489802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.490164 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.490625 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.490626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.490627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.490767 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.491198 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.491205 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.491391 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.491566 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.491758 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":119,"threads_started":1}
I20260811 18:52:39.492112 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.483773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60538)
0811 18:52:39.483825 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.483835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.483867 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.484223 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.484224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.484379 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.484415 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.484418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.485379 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.485380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.486181 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.486183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.486887 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.486888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.486947 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.487388 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.487393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.488875 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.490961 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.490962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.490963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.491228 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.491480 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.491481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.491481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.491544 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.491851 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.491852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.491898 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.491940 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.492032 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 18:52:39.492537 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.483937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.9.215.60:34891)
0811 18:52:39.484094 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.484096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.484103 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:52:39.484120 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.484121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.484130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.484198 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.484496 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.485276 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.486284 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.486794 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.486872 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.488959 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.488970 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.488971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.488978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.490899 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.491310 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.491314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.491314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.491384 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.491612 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.491613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.491613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.491772 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.492243 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.492294 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.492390 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260811 18:52:39.492583 22954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.493796 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.483186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.9.215.60:34891)
0811 18:52:39.483393 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.483397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.483416 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:52:39.483438 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.483440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.483486 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.483563 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.484316 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.485273 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.486284 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.486799 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.486872 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.489557 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.489558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.489558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.489565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.492479 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.492972 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.492973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.492973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.493031 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.493211 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.493212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.493213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.493395 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.493531 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.493613 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.493690 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":174,"thread_start_us":133,"threads_started":1}
I20260811 18:52:39.494771 22665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3: Bootstrap starting.
I20260811 18:52:39.495769 22665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:39.496126 22665 log.cc:824] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:39.497051 22665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3: No bootstrap required, opened a new log
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.62@KRBTEST.COM
I20260811 18:52:39.499009 22665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.499154 22665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e83be23305e409d81a0a865cd4fd3b3, State: Initialized, Role: FOLLOWER
I20260811 18:52:39.499238 22665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.499581 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.499661 22665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:39.499675 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:39.499748 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.492896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.9.215.61:34073)
0811 18:52:39.492947 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.492949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.492991 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:52:39.493063 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.493064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.493077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.493125 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.493405 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.494057 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.494883 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.495185 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.495260 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.497251 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.497253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.497253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.497261 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.498681 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.498991 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.498993 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.498993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.499050 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.499210 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.499211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.499212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.499328 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.499389 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.499574 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.499657 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 18:52:39.499987 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.492836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.61:34073 (local address 127.0.0.1:35672)
0811 18:52:39.492941 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.492951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.492995 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.493126 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.493127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.493277 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.493316 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.493317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.494125 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.494126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.494782 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.494784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.495267 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.495268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.495343 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.495778 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.495783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.497143 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.498765 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.498766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.498766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.498929 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.499102 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.499103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.499103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.499160 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.499380 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.499381 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.499427 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.499821 (+ 394us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.499925 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 18:52:39.500393 22954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:52:39.502763 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.494749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.9.215.62:37739)
0811 18:52:39.494782 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.494784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.494792 (+ 8us) server_negotiation.cc:407] Connection header received
0811 18:52:39.494845 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.494846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.494855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.494908 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.495492 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.496036 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.496829 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.497026 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.497078 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.499584 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.499586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.499586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.499594 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.501168 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.501729 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.501730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.501730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.501797 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.502036 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.502037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.502037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.502179 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.502361 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.502563 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.502644 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 18:52:39.502698 22993 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:39.502863 22993 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.60@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:52:39.503710 22536 master_runner.cc:186] Error getting master registration for 127.9.215.61:34073: OK, instance_id { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" instance_seqno: 1786474359306188 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:52:39.506683 22536 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:52:39.507119 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.500661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60552)
0811 18:52:39.500740 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.500749 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.500788 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.500888 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.500889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.501035 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.501076 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.501079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.501699 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.501700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.502409 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.502412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.502881 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.502881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.502925 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.503391 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.503396 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.504661 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.506102 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.506103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.506104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.506266 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.506428 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.506428 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.506429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.506481 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.506801 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.506802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.506842 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.506897 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.507007 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 18:52:39.507126 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.500701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.9.215.60:34891)
0811 18:52:39.500746 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.500748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.500775 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:52:39.500835 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.500836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.500844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.500887 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.501139 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.501630 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.502486 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.502775 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.502834 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.504741 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.504742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.504743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.504750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.506027 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.506325 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.506326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.506327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.506377 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.506530 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.506530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.506531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.506721 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.506781 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.506935 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.507050 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":24}
I20260811 18:52:39.508740 22954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: Bootstrap starting.
I20260811 18:52:39.509433 22954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:39.509770 22954 log.cc:824] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:39.510840 22954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: No bootstrap required, opened a new log
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.215.61@KRBTEST.COM
I20260811 18:52:39.512933 22954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.513088 22954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd77eb92cd6485bbc1ec99115872f44, State: Initialized, Role: FOLLOWER
I20260811 18:52:39.513185 22954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.513562 22954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:39.513772 22997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.513892 22997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:52:39.515939 23016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:52:39.516009 23016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:52:39.516156 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.503717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.9.215.61:34073)
0811 18:52:39.503753 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.503755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.506859 (+ 3104us) server_negotiation.cc:407] Connection header received
0811 18:52:39.506960 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.506962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.506974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.507004 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.507695 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.508363 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.509369 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.509658 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.509742 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.512670 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.512672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.512672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.512682 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.514509 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.515099 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.515101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.515101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.515188 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.515425 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.515426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.515426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.515604 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.515738 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.515968 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.516061 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 18:52:39.518061 10076 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:52:39.519132 22619 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4e83be23305e409d81a0a865cd4fd3b3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53534
I20260811 18:52:39.519309 22619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:52:39.519358 22619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:39.517464 22829 master_runner.cc:186] Error getting master registration for 127.9.215.61:34073: OK, instance_id { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" instance_seqno: 1786474359306188 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:52:39.519894 22829 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:52:39.520346 22619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.520547 22619 leader_election.cc:288] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [CANDIDATE]: Term 1 election: Requested vote from peers 6750d0a9e39742aab163fdc117751dc8 (127.9.215.61:34073), 3fd77eb92cd6485bbc1ec99115872f44 (127.9.215.60:34891)
I20260811 18:52:39.521162 22911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e83be23305e409d81a0a865cd4fd3b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3fd77eb92cd6485bbc1ec99115872f44"
I20260811 18:52:39.521298 22911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:39.522190 22911 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e83be23305e409d81a0a865cd4fd3b3 in term 1.
I20260811 18:52:39.522408 22573 leader_election.cc:302] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd77eb92cd6485bbc1ec99115872f44, 4e83be23305e409d81a0a865cd4fd3b3; no voters:
I20260811 18:52:39.522585 22966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:52:39.522768 22966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [term 1 LEADER]: Becoming Leader. State: Replica: 4e83be23305e409d81a0a865cd4fd3b3, State: Running, Role: LEADER
I20260811 18:52:39.522877 22966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.523392 23028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e83be23305e409d81a0a865cd4fd3b3. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.523636 23028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:39.527763 23030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:52:39.528040 23030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:52:39.529997 22911 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [term 1 FOLLOWER]: Refusing update from remote peer 4e83be23305e409d81a0a865cd4fd3b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:39.530505 23028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:39.532148 22997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e83be23305e409d81a0a865cd4fd3b3. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.532279 22997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:39.540987 23028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.541121 23028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:39.541206 23031 mvcc.cc:205] Tried to move back new op lower bound from 7317398976631558144 to 7317398976606461952. Current Snapshot: MvccSnapshot[applied={T|T < 7317398976631558144}]
I20260811 18:52:39.541790 23033 mvcc.cc:205] Tried to move back new op lower bound from 7317398976631558144 to 7317398976606461952. Current Snapshot: MvccSnapshot[applied={T|T < 7317398976631558144}]
I20260811 18:52:39.541885 22997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.541957 22997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:39.542054 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.531930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.9.215.61:34073)
0811 18:52:39.531978 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.531979 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.533012 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 18:52:39.533213 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.533215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.533228 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.533261 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.534253 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.535025 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.536153 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.536469 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.536514 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.538506 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.538508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.538508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.538518 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.540301 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.540936 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.540937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.540938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.541003 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.541397 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.541398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.541398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.541551 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.541624 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.541849 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.541943 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:39.544589 22815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } attempt: 3
Aug 11 18:52:39 dist-test-slave-pww0 krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474359, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.215.61@KRBTEST.COM for kudu/127.9.215.60@KRBTEST.COM
I20260811 18:52:39.551414 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.532940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.9.215.60:34891)
0811 18:52:39.532982 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.532984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.542751 (+ 9767us) server_negotiation.cc:407] Connection header received
0811 18:52:39.542852 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.542853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.542865 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.542894 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.543566 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.544315 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.545409 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.545709 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.545748 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.547441 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.547442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.547443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.547450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.549333 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.549879 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.549881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.549881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.549959 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.550444 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.550445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.550445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.550610 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.550703 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.551195 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.551306 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:39.552883 23030 catalog_manager.cc:1347] Generated new cluster ID: 0be3f4e38ab24e6588bdca09a0bc2db3
I20260811 18:52:39.552951 23030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:52:39.553702 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.544862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.215.60:34891 (local address 127.0.0.1:60578)
0811 18:52:39.544947 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 18:52:39.544957 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:52:39.544992 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:52:39.545202 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:52:39.545203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:52:39.545361 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:52:39.545405 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.545407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.546238 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.546239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.547047 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:52:39.547050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.547693 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.547694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:52:39.547786 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.548165 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:52:39.548170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:52:39.549741 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:52:39.552425 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.552427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.552427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.552665 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.552933 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:52:39.552934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:52:39.552934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:52:39.553017 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:52:39.553365 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:52:39.553367 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:52:39.553434 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 18:52:39.553489 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 18:52:39.553599 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 18:52:39.554222 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.545059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.9.215.60:34891)
0811 18:52:39.545095 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.545096 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.545103 (+ 7us) server_negotiation.cc:407] Connection header received
0811 18:52:39.545121 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.545122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.545130 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.545169 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.545479 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.546150 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.547171 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.547552 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.547614 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.549856 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.549858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.549858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.549883 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.552301 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.552768 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.552769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.552769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.552848 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.553097 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.553098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.553099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.553270 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.553327 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.554041 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.554132 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:52:39.556921 22815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: Bootstrap starting.
I20260811 18:52:39.557721 22815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: Neither blocks nor log segments found. Creating new log.
I20260811 18:52:39.558039 22815 log.cc:824] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:52:39.559078 22815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: No bootstrap required, opened a new log
I20260811 18:52:39.561839 22815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.562007 22815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6750d0a9e39742aab163fdc117751dc8, State: Initialized, Role: FOLLOWER
I20260811 18:52:39.563234 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.532252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.9.215.62:37739)
0811 18:52:39.532306 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.532308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.540333 (+ 8025us) server_negotiation.cc:407] Connection header received
0811 18:52:39.540447 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.540448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.540461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.540509 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.543753 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.544425 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.545476 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.545757 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.545800 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.552432 (+ 6632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:52:39.552434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:52:39.552435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:52:39.552443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:52:39.560904 (+ 8461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.561776 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.561778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.561779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.561854 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:52:39.562391 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:52:39.562392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:52:39.562392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:52:39.562539 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:52:39.562624 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.562997 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.563125 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:52:39.563831 23030 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:52:39.564587 23030 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:52:39.570466 22815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } }
I20260811 18:52:39.571075 23046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.571213 22767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:52:39.573470 22767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [term 1 FOLLOWER]: Refusing update from remote peer 4e83be23305e409d81a0a865cd4fd3b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:52:39.573459 23030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3: Generated new TSK 0
I20260811 18:52:39.573661 23030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:52:39.571245 22768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e83be23305e409d81a0a865cd4fd3b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6750d0a9e39742aab163fdc117751dc8"
I20260811 18:52:39.571082 22815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:52:39.574652 22768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e83be23305e409d81a0a865cd4fd3b3 in term 1.
I20260811 18:52:39.575067 22966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:52:39.577869 23080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e83be23305e409d81a0a865cd4fd3b3. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.577998 23080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:39.578246 23085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:39.578323 23085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:39.578361 23046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:52:39.580442 23078 catalog_manager.cc:1259] Loaded cluster ID: 0be3f4e38ab24e6588bdca09a0bc2db3
I20260811 18:52:39.580559 23078 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: loading cluster ID for follower catalog manager: success
I20260811 18:52:39.581823 23078 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: acquiring CA information for follower catalog manager: success
I20260811 18:52:39.582258 23078 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6750d0a9e39742aab163fdc117751dc8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:52:39.613217 22673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.611710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.9.215.62:37739)
0811 18:52:39.611752 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.611754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.612506 (+ 752us) server_negotiation.cc:407] Connection header received
0811 18:52:39.612603 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.612604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.612616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.612643 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.613138 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53552: BlockingRecv error: recv got EOF from 127.0.0.1:53552 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 18:52:39.614292 22821 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.611477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.9.215.61:34073)
0811 18:52:39.611530 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.611532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.613658 (+ 2126us) server_negotiation.cc:407] Connection header received
0811 18:52:39.613751 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.613752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.613763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.613791 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.614210 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35700: BlockingRecv error: recv got EOF from 127.0.0.1:35700 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 18:52:39.615171 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.612216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.9.215.60:34891)
0811 18:52:39.612270 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.612271 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.614618 (+ 2347us) server_negotiation.cc:407] Connection header received
0811 18:52:39.614704 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.614705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.614716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.614744 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.615109 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60584: BlockingRecv error: recv got EOF from 127.0.0.1:60584 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:52:39.620685 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.617097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.9.215.61:34073)
0811 18:52:39.617138 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.617139 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.617392 (+ 253us) server_negotiation.cc:407] Connection header received
0811 18:52:39.617504 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.617505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.617555 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.617601 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.617874 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.618615 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.619702 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.619984 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.620026 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.620289 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.620420 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.620473 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.620583 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.620606 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 18:52:39.624357 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.623364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.9.215.60:34891)
0811 18:52:39.623410 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.623412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.623705 (+ 293us) server_negotiation.cc:407] Connection header received
0811 18:52:39.623821 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.623823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.623844 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.623877 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.624293 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60600: BlockingRecv error: recv got EOF from 127.0.0.1:60600 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:39.631136 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.627109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.9.215.61:34073)
0811 18:52:39.627156 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.627157 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.627530 (+ 373us) server_negotiation.cc:407] Connection header received
0811 18:52:39.627639 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.627640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.627652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.627680 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.627870 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.628944 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.630105 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.630408 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.630471 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.630833 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.630873 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.630900 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.631003 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.631030 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 18:52:39.635082 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.634172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.9.215.60:34891)
0811 18:52:39.634232 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.634234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.634401 (+ 167us) server_negotiation.cc:407] Connection header received
0811 18:52:39.634505 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.634507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.634520 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.634553 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.634981 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60604: BlockingRecv error: recv got EOF from 127.0.0.1:60604 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:52:39.642617 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.638866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35734 (local address 127.9.215.61:34073)
0811 18:52:39.638917 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.638920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.639513 (+ 593us) server_negotiation.cc:407] Connection header received
0811 18:52:39.639634 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.639635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.639649 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.639679 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.639894 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.640621 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.641761 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.642042 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.642104 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.642333 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.642374 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.642402 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.642508 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.642531 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 18:52:39.646540 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.645558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.9.215.60:34891)
0811 18:52:39.645604 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.645606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.645908 (+ 302us) server_negotiation.cc:407] Connection header received
0811 18:52:39.646016 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.646017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.646030 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.646060 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.646470 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60618: BlockingRecv error: recv got EOF from 127.0.0.1:60618 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 18:52:39.656116 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.652131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.9.215.61:34073)
0811 18:52:39.652196 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.652198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.652525 (+ 327us) server_negotiation.cc:407] Connection header received
0811 18:52:39.652641 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.652643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.652654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.652684 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.652890 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.653639 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.654907 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.655323 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.655421 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.655771 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.655830 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.655860 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.656002 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.656029 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 18:52:39.659942 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.658954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.9.215.60:34891)
0811 18:52:39.659000 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.659002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.659272 (+ 270us) server_negotiation.cc:407] Connection header received
0811 18:52:39.659390 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.659392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.659405 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.659437 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.659877 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60630: BlockingRecv error: recv got EOF from 127.0.0.1:60630 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 18:52:39.673017 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.669453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.9.215.61:34073)
0811 18:52:39.669501 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.669503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.669801 (+ 298us) server_negotiation.cc:407] Connection header received
0811 18:52:39.669910 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.669911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.669923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.669953 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.670175 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.670942 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.672166 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.672484 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.672529 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.672738 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.672775 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.672804 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.672904 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.672927 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 18:52:39.676856 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.675801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.9.215.60:34891)
0811 18:52:39.675849 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.675852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.676177 (+ 325us) server_negotiation.cc:407] Connection header received
0811 18:52:39.676333 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.676334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.676348 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.676377 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.676788 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60646: BlockingRecv error: recv got EOF from 127.0.0.1:60646 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 18:52:39.697671 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.694358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35772 (local address 127.9.215.61:34073)
0811 18:52:39.694403 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.694405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.694699 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:52:39.694794 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.694795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.694806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.694833 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.695011 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.695792 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.696889 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.697187 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.697227 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.697417 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.697450 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.697473 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.697565 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.697585 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 18:52:39.701154 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.700110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.9.215.60:34891)
0811 18:52:39.700164 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.700165 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.700420 (+ 255us) server_negotiation.cc:407] Connection header received
0811 18:52:39.700512 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.700513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.700525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.700552 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.701079 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60658: BlockingRecv error: recv got EOF from 127.0.0.1:60658 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 18:52:39.737396 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.734616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.9.215.61:34073)
0811 18:52:39.734680 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.734682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.734766 (+ 84us) server_negotiation.cc:407] Connection header received
0811 18:52:39.734803 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.734804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.734816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.734896 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.735024 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.735787 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.736670 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.736983 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.737059 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.737162 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.737198 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.737242 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.737287 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.737309 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 18:52:39.739960 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.739234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.9.215.60:34891)
0811 18:52:39.739279 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.739281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.739397 (+ 116us) server_negotiation.cc:407] Connection header received
0811 18:52:39.739476 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.739477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.739489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.739551 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.739895 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60666: BlockingRecv error: recv got EOF from 127.0.0.1:60666 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:52:39.807276 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.805128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.9.215.61:34073)
0811 18:52:39.805181 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.805183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.805300 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:52:39.805336 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.805337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.805348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.805413 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.805527 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.806007 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.806741 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.806932 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.806987 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.807083 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.807112 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.807163 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.807197 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.807213 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 18:52:39.809708 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.809123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.9.215.60:34891)
0811 18:52:39.809191 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.809193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.809225 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:52:39.809282 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.809282 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.809293 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.809341 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.809643 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60678: BlockingRecv error: recv got EOF from 127.0.0.1:60678 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:52:39.941702 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:39.939066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.9.215.61:34073)
0811 18:52:39.939129 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.939131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.939146 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:52:39.939230 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.939231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.939241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.939306 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:39.939570 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.940097 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.940900 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:39.941095 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:39.941160 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:39.941375 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:39.941412 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:39.941467 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:39.941594 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 18:52:39.941615 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 18:52:39.944290 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:39.943643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.9.215.60:34891)
0811 18:52:39.943707 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:39.943709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:39.943822 (+ 113us) server_negotiation.cc:407] Connection header received
0811 18:52:39.943853 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:39.943854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:39.943866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:39.943923 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:39.944231 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60694: BlockingRecv error: recv got EOF from 127.0.0.1:60694 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:52:40.128660 23282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e83be23305e409d81a0a865cd4fd3b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e83be23305e409d81a0a865cd4fd3b3" member_type: VOTER last_known_addr { host: "127.9.215.62" port: 37739 } } peers { permanent_uuid: "6750d0a9e39742aab163fdc117751dc8" member_type: VOTER last_known_addr { host: "127.9.215.61" port: 34073 } } peers { permanent_uuid: "3fd77eb92cd6485bbc1ec99115872f44" member_type: VOTER last_known_addr { host: "127.9.215.60" port: 34891 } } }
I20260811 18:52:40.128837 23282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83be23305e409d81a0a865cd4fd3b3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:52:40.203892 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.201562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35816 (local address 127.9.215.61:34073)
0811 18:52:40.201616 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.201618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.201628 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:40.201722 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.201723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.201734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.201780 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:40.201913 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.202431 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.203209 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.203411 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.203502 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.203645 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:40.203682 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:40.203722 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.203784 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.203805 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 18:52:40.206732 22964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:52:40.205974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.9.215.60:34891)
0811 18:52:40.206046 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.206049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.206059 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:40.206086 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.206086 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.206096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.206157 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:52:40.206668 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60700: BlockingRecv error: recv got EOF from 127.0.0.1:60700 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:52:40.516500 23016 catalog_manager.cc:1259] Loaded cluster ID: 0be3f4e38ab24e6588bdca09a0bc2db3
I20260811 18:52:40.516604 23016 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: loading cluster ID for follower catalog manager: success
I20260811 18:52:40.517472 23016 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: acquiring CA information for follower catalog manager: success
I20260811 18:52:40.517822 23016 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3fd77eb92cd6485bbc1ec99115872f44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:52:40.722505 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.720111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.9.215.61:34073)
0811 18:52:40.720240 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.720242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.720254 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:52:40.720318 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.720320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.720333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.720389 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:40.720515 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.721029 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.721823 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.722032 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.722088 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.722237 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:40.722273 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:40.722312 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.722377 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.722406 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260811 18:52:40.727378 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.724618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.9.215.60:34891)
0811 18:52:40.724776 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.724778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.724788 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:52:40.724851 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.724852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.724903 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.724966 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:40.725059 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.725558 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.726401 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.726617 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.726688 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.726907 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:40.727032 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:40.727113 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.727224 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.727268 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260811 18:52:40.732211 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.729243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.9.215.62:37739)
0811 18:52:40.729506 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.729508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.729514 (+ 6us) server_negotiation.cc:407] Connection header received
0811 18:52:40.729552 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.729553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.729594 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.729698 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:40.729835 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.730573 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.731318 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.731588 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.731654 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.731840 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:40.731962 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:40.732026 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.732084 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.732111 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":49,"threads_started":1}
I20260811 18:52:40.732307 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.729397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.9.215.60:34891)
0811 18:52:40.729434 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.729436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.729867 (+ 431us) server_negotiation.cc:407] Connection header received
0811 18:52:40.729891 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.729892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.729899 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.729939 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:40.730047 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.730567 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.731590 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.731876 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.731935 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.732069 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:40.732103 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:40.732149 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.732204 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.732231 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 18:52:40.732548 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:52:40.729228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.9.215.61:34073)
0811 18:52:40.729277 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 18:52:40.729278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 18:52:40.729392 (+ 114us) server_negotiation.cc:407] Connection header received
0811 18:52:40.729499 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:52:40.729500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:52:40.729512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:52:40.729558 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:52:40.729661 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.730400 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.731517 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:52:40.731809 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:52:40.731868 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:52:40.732261 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:52:40.732301 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:52:40.732346 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:52:40.732447 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 18:52:40.732476 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:52:40.734148 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22536
I20260811 18:52:40.751057 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22680
I20260811 18:52:40.768664 10076 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdX0PXQ/build/release/bin/kudu with pid 22829
2026-08-11T18:52:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3169 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6273 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292700 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST